CN112905304B

CN112905304B - Communication method and device between virtual machines, physical host and medium

Info

Publication number: CN112905304B
Application number: CN202110250891.6A
Authority: CN
Inventors: 李波; 朱民
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-03-08
Filing date: 2021-03-08
Publication date: 2024-07-09
Anticipated expiration: 2041-03-08
Also published as: CN112905304A

Abstract

The application discloses a communication method, a device, a physical host and a medium between virtual machines, wherein the method comprises the following steps: when the device driver corresponding to the first virtual machine is monitored to write an I/O notification register in PCIe device corresponding to the first virtual machine, reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine; determining the IP address of a second physical host of the receiver based on a second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table; and sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier. Therefore, the method has strong adaptability, does not need to reserve an IP address field, and is flexible to use.

Description

Communication method and device between virtual machines, physical host and medium

Technical Field

The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for communication between virtual machines, a physical host, and a medium.

Background

With the improvement of hardware performance, the computing power of the physical host is greatly improved. However, due to uneven use of resources by the operating system, hardware performance is not fully utilized, resulting in serious waste of computing resources. In this case, the virtualization technology has been developed, so that multiple virtual machines that are independent of each other can be simultaneously run on the same physical host, where each virtual machine has a complete hardware device, including BIOS (Basic Input Output System ), CPU, memory, disk, network card, and so on. The cloud computing takes the virtualization technology as a core, provides elastic resources for users, greatly improves the resource utilization rate of the physical host, and reduces the cost of the users. In the current cloud computing platform, each virtual machine is generally only provided with one network card for network communication of a user. However, in reality, many scenarios require data transmission between virtual machines on different physical hosts, and it is required to ensure that user traffic on the virtual machines is not affected. For example, the service provider's management platform needs to manage the business virtual machines of the customers, and even needs to transmit some tools inside the virtual machines, such as VMtools (Virtual Machine Tools ), security tools.

To meet the requirements in the above examples, in the current KVM (Kernel-based Virtual Machine, open source virtual machine) platform, there are two communication modes between virtual machines on different physical hosts: the first is serial port-based data communication, which needs to directly specify the physical host IP (Internet Protocol ) address of the other party in the virtual machines of both communication parties, and once the virtual machines migrate, the physical host IP addresses configured in each virtual machine in communication with the virtual machines need to be directly modified, so the cloud computing platform with large scale, strong flexibility and dynamic automation configuration has poor adaptability. And secondly, network-based data communication, wherein the network-based data communication needs to configure a special network card and reserve an IP section for each virtual machine, is complex in management, needs participation of users, and is inflexible in use.

Disclosure of Invention

In view of the above, the present application aims to provide a virtual machine communication method, device, equipment, and medium, which can adapt to a cloud computing platform with large scale, strong flexibility, and dynamic automation configuration, has strong adaptability, does not need to reserve an IP address field, and is flexible to use. The specific scheme is as follows:

in a first aspect, the present application discloses a communication method between virtual machines, which is applied to a first physical host running a first virtual machine, and includes:

When the device driver corresponding to the first virtual machine is monitored to write an I/O notification register in PCIe device corresponding to the first virtual machine, reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine;

Determining the IP address of a second physical host of the receiver based on the second virtual machine identification in the original data packet to be transmitted and a preset virtual machine identification and server IP comparison table;

And sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

Optionally, the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host includes:

Encapsulating the original data packet to be transmitted into a network data packet to be transmitted based on the IP address of the second physical host;

And sending the network data packet to be transmitted to the second physical host, wherein the network data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier after being analyzed, and the second virtual machine operates on the second physical host.

Optionally, before the step of reading the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine when the device driver corresponding to the first virtual machine is monitored to perform a write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the method further includes:

When the first virtual machine is created on the first physical host, determining the target shared memory from a local preset shared memory body, and distributing the target shared memory to the first virtual machine;

And distributing an event notification file for the first virtual machine, and storing a first virtual machine identifier of the first virtual machine, memory use information of the target shared memory and a file identifier of the event notification file into a preset memory use information table.

Optionally, the determining the target shared memory from the local preset shared memory bank further includes:

and dynamically expanding the preset shared memory body when the residual memory space of the preset shared memory body is smaller than the memory size required by the first virtual machine.

Optionally, after storing the first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and the file identifier of the event notification file in a preset memory usage information table, the method further includes:

And sending the memory use information of the target shared memory and the event notification file to PCIe equipment corresponding to the first virtual machine.

Receiving a virtual machine identifier sent by a cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier;

and constructing a virtual machine identifier and server IP comparison table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

Optionally, after the virtual machine identifier and the server IP lookup table are constructed based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier, the method further includes:

creating a local cache table corresponding to the virtual machine identifier and the server IP comparison table;

And storing the table entry of which the use times exceeds the preset times in the past preset time in the virtual machine identification and server IP comparison table into the local cache table.

Optionally, after the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, the method further includes:

Receiving a confirmation network data packet sent by the second physical host, wherein the confirmation network data packet is obtained by the second physical host by packaging a confirmation original data packet based on a first virtual machine identifier in the confirmation original data packet, and the confirmation original data packet is used for confirming whether the second virtual machine receives the original data packet to be transmitted or not;

analyzing the confirmation network data packet to obtain the confirmation original data packet;

and transmitting the original acknowledgement data packet to the first virtual machine based on the first virtual machine identification in the original acknowledgement data packet.

Optionally, the method further comprises:

Receiving network data packets to be processed sent by other physical hosts except the first physical host;

analyzing the network data packet to be processed to obtain a raw data packet to be processed corresponding to the network data packet to be processed;

Storing the original data packet to be processed into a shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the original data packet to be processed;

and inserting a corresponding target virtual interrupt into the target virtual machine, wherein the target virtual interrupt is used for triggering a target device driver corresponding to the target virtual machine to receive the original data packet to be processed.

Optionally, the storing the to-be-processed original data packet in the shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the to-be-processed original data packet includes:

Determining a shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the original data packet to be processed;

and storing the original data packet to be processed into a write ring buffer queue in a shared memory corresponding to the destination virtual machine identifier.

In a second aspect, the present application discloses an inter-virtual machine communication device, applied to a first physical host running a first virtual machine, comprising:

The data reading module is used for reading an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine when the device driver corresponding to the first virtual machine is monitored to write an I/O notification register in PCIe device corresponding to the first virtual machine;

the IP address determining module is used for determining the IP address of the second physical host of the receiver based on the second virtual machine identification in the original data packet to be transmitted and a preset virtual machine identification and server IP comparison table;

The data packet sending module is configured to send the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, where the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

In a third aspect, the present application discloses a physical host comprising:

A memory and a processor;

Wherein the memory is used for storing a computer program;

the processor is configured to execute the computer program to implement the disclosed method for communication between virtual machines.

In a fourth aspect, the present application discloses a computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the previously disclosed inter-virtual machine communication method.

When the device driver corresponding to the first virtual machine is monitored to write the I/O notification register in the PCIe device corresponding to the first virtual machine, the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine is read, then the IP address of a second physical host of a receiver is determined based on a second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table, and then the original data packet to be transmitted can be sent to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to the second virtual machine corresponding to the second virtual machine identifier. Therefore, in the application, when the device driver corresponding to the first virtual machine monitors that the device driver corresponding to the first virtual machine performs write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the original data packet to be transmitted can be read out from the shared memory corresponding to the first virtual machine, and the PI address of the second physical host of the receiver is determined based on the first virtual machine identifier and the preset virtual machine identifier and server IP address comparison table in the original data packet to be transmitted, the original data packet to be transmitted is sent to the second physical host based on the IP address of the second physical host, the second physical host can transmit the original data packet to the second virtual machine identifier corresponding to the second virtual machine, so that only the preset virtual machine identifier and the server IP address comparison table are required to be maintained, the host IP address of the communication is found according to the virtual machine identifier and the server IP address comparison table, the host can be adapted to large-scale, the virtual machine can be configured automatically, the virtual machine is not required to be opened by the virtual machine identifier and the virtual machine, and the virtual machine is required to be opened by the virtual machine.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a communication method between virtual machines disclosed by the application;

FIG. 2 is a partial flow chart of a specific method for communication between virtual machines according to the present application;

FIG. 3 is a partial flow chart of a specific method for communication between virtual machines according to the present disclosure;

FIG. 4 is a partial flow chart of a specific method for communication between virtual machines according to the present application;

FIG. 5 is a flowchart of a specific communication method between virtual machines according to the present disclosure;

FIG. 6 is a schematic diagram of a communication device between virtual machines according to the present application;

FIG. 7 is a schematic diagram of a method for high-speed communication between virtual machine processes across physical machines in a cloud virtualization platform according to the present application;

FIG. 8 is a diagram illustrating a shared memory management scheme according to the present disclosure;

FIG. 9 is a schematic diagram of initialization of a PCIe device in accordance with the present disclosure;

fig. 10 is a schematic diagram of a physical host structure according to the present disclosure.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

Referring to fig. 1, an embodiment of the present application discloses a communication method between virtual machines, which is applied to a first physical host running a first virtual machine, and the method includes:

step S11: when the device driver corresponding to the first virtual machine is monitored to write the I/O notification register in the PCIe device corresponding to the first virtual machine, the original data packet to be transmitted in the target shared memory corresponding to the first virtual machine is read.

In a specific implementation process, when it is monitored that a device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in a PCIe (PERIPHERAL COMPONENT INTERCONNECT EXPRESS, high-speed peripheral component interconnect standard) device corresponding to the first virtual machine, an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine is read, where the original data packet to be transmitted is a data packet written to a PCIe memory in the PCIe device after the first virtual machine establishes a connection with the PCIe device through a virtual socket, and the PCIe memory is a mapping in the target shared memory.

That is, when the first virtual machine on the first physical host needs to transmit data to the second virtual machine on the second physical host outside the first physical host, the first virtual machine first establishes a connection with a PCIe device corresponding to the first virtual machine through a virtual socket (vsocket), writes an original data packet to be transmitted into a PCIe memory in the PCIe device, and performs a write operation on an I/O (Input/Output) notification register in the PCIe device through a device driver corresponding to the first virtual machine, at this time, the original data packet to be transmitted can be read from a target shared memory corresponding to the first virtual machine by the first physical host, where the PCIe memory is a mapping in the PCIe device. Wherein vsocket is a virtual socket, an anonymous file handle, which is provided for the application.

Therefore, the virtual machine which needs to send data only needs to write the original data packet to be transmitted into the PCIe memory in the corresponding PCIe device, and the physical host can read the original data packet to be transmitted from the shared memory mapped to the PCIe memory, so that zero-copy data transmission is realized between the virtual machine and the physical host through the shared memory, and the data transmission efficiency is improved.

When the cloud platform creates a virtual machine, the cloud platform management system assigns it a unique identification (virtual MACHINE IDENTITY, VMID, virtual machine identification) that accompanies the entire life cycle of the virtual machine and is only reclaimed when the virtual machine is destroyed. The IP addresses of the physical hosts are fixed, and the information synchronization is realized among the physical hosts through network communication.

In order for the user layer application to use the device driver, the device driver encapsulates the PCIe device into a character device and provides vsocket to implement multi-process concurrent read-write data. The device driver also provides a set of kernel mode interfaces for the kernel module to directly call.

Specifically, an application program in the first virtual machine can acquire the own first virtual machine identifier VMID through a register on the PCIe device, and can set the second virtual machine identifier VMID of the opposite communication end through a program parameter. Creating vsocket through the first virtual machine identifier VMID, the sender port, the second virtual machine identifier VMID and the receiver port, and directly writing the original data packet to be transmitted into a PCIe memory of a PCIe device corresponding to the first virtual machine. By the design, the virtual socket is only related to the sender virtual machine identification, the sender port, the receiver virtual machine identification and the receiver port, and once an application program is started, the application program cannot be changed due to migration of the virtual machine.

When the first virtual machine writes the original data packet to be transmitted into the PCIe memory, a device driver corresponding to the first virtual machine triggers an interrupt request, and writes an I/O notification register in the PCIe device, at this time, the first physical host is triggered to read the original data packet to be transmitted from a target shared memory corresponding to the first virtual machine. Because the PCIe memory in the PCIe device is the mapping in the PCIe device of the target shared memory corresponding to the first virtual machine, the first physical host can directly read the original data packet to be transmitted from the target shared memory, zero copy data transmission between the first virtual machine and the first physical host is realized through the shared memory, and the data transmission efficiency is improved.

The target shared memory comprises a read ring queue and a write ring queue. Each ring queue has two pointers front and end. The sender operates end, and the receiver operates front, so that the read-write annular queue is realized in a lock-free mode, and the read-write speed is more efficient than that of a lock mechanism.

Step S12: and determining the IP address of the second physical host of the receiver based on the second virtual machine identification in the original data packet to be transmitted and a preset virtual machine identification and server IP comparison table.

It can be understood that after the original data packet to be transmitted is read, the IP address of the second physical host of the receiver needs to be determined based on the second virtual machine identifier in the original data packet to be transmitted and the preset virtual machine identifier and the server IP address comparison table, where the second virtual machine corresponding to the second virtual machine identifier is the receiver virtual machine running on the second physical host.

Specifically, after the original data packet to be transmitted is read out, the original data packet to be transmitted is analyzed, and according to a second virtual machine identifier in a packet header of the original data packet to be transmitted, a preset virtual machine identifier and server IP address comparison table is searched to obtain an IP address of a second physical host of a receiver corresponding to the second virtual machine identifier.

In an actual application, before the device driver corresponding to the first virtual machine is monitored to perform a write operation on the I/O notification register in the PCIe device corresponding to the first virtual machine, the method further includes: receiving a virtual machine identifier sent by a cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier; and constructing a virtual machine identifier and server IP comparison table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

That is, the cloud platform management system to which the first virtual machine belongs may send each virtual machine identifier in the entire cloud platform and a physical host IP address of a host corresponding to each virtual machine identifier to each physical host on the cloud platform, where the first physical host needs to obtain the virtual machine identifier sent by the cloud platform management system and a physical host IP address corresponding to the virtual machine identifier, and construct the virtual machine identifier and server IP lookup table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

In practical application, the cloud platform management system needs to be expanded in a cloud platform virtualization system, so that the cloud platform virtualization system can dynamically intercept operations such as starting, migration and closing of a virtual machine, and when a new virtual machine is created, the VMID and an IP address of a physical host where the virtual machine is located are synchronized to each physical host of the cloud platform, so that a virtual machine identifier and a server IP comparison table can be updated. When the virtual machine is migrated, all physical hosts are notified, and the virtual machine identification and the server IP comparison table are updated, so that timeliness of information is guaranteed.

After the virtual machine identifier and the server IP comparison table are constructed, a local cache table corresponding to the virtual machine identifier and the server IP comparison table can be also created; and storing the table entry of which the use times exceeds the preset times in the past preset time in the virtual machine identification and server IP comparison table into the local cache table.

That is, a local cache table may be created for the virtual machine identifier and server IP lookup table, and entries in the virtual machine identifier and server IP lookup table that are used more than a preset number of times within a preset time period in the past are stored in the local cache table. Therefore, frequently used table items in the virtual machine identification and server IP comparison table can be stored in the local cache table, quick inquiry can be realized, and data transmission efficiency is improved.

Step S13: and sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

After the IP address of the second physical host is obtained, the to-be-transmitted original data packet needs to be sent to the second physical host based on the IP address of the second physical host, and after the to-be-transmitted original data packet is received by the second physical host, the to-be-transmitted original data packet is transmitted to a second virtual machine corresponding to the second virtual machine identifier.

The sending the to-be-transmitted original data packet to the second physical host based on the IP address of the second physical host may specifically include: encapsulating the original data packet to be transmitted into a network data packet to be transmitted based on the IP address of the second physical host; and sending the network data packet to be transmitted to the second physical host, wherein the network data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier after being analyzed, and the second virtual machine operates on the second physical host.

That is, the original data packet to be transmitted is encapsulated into a network data packet to be transmitted based on the IP address of the second physical host, then the network data packet to be transmitted is sent to the second physical host, after the second physical host receives the network data packet to be transmitted, the network data packet to be transmitted is parsed, the original data packet to be transmitted is obtained, and the original data packet to be transmitted is transmitted to the second virtual machine according to the second virtual machine identifier in the original data packet to be transmitted.

Referring to fig. 2, when it is monitored that the device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in the PCIe device corresponding to the first virtual machine, before reading an original data packet to be transmitted in the target shared memory corresponding to the first virtual machine, the method further includes:

step S21: and when the first virtual machine is created on the first physical host, determining the target shared memory from a local preset shared memory body, and distributing the target shared memory to the first virtual machine.

When the device driver corresponding to the first virtual machine is monitored to perform writing operation on an I/O notification register in PCIe device corresponding to the first virtual machine, before an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine is read, the first virtual machine is created on the first physical host, and relevant configuration is performed, so that the first virtual machine and the first physical host can achieve the virtual machine communication method.

Specifically, when the first virtual machine needs to be created on the first physical host, determining the target shared memory from a local preset shared memory body, and distributing the target shared memory to the first virtual machine.

In the process of determining the target shared memory from the local preset shared memory bank, the method further comprises: and dynamically expanding the preset shared memory body when the residual memory space of the preset shared memory body is smaller than the memory size required by the first virtual machine. That is, when the remaining memory space in the preset shared memory body is insufficient, the memory size of the preset shared memory body can be dynamically enlarged, so that the subsequently started virtual machine has enough resources to create a channel between the virtual machine and the first physical host based on the shared memory.

When the first virtual machine is created on the first physical host, the first physical host also receives the first virtual machine identifier and the IP address of the first physical host, which are sent by the cloud platform management system, and updates the virtual machine identifier and the IP address of the first physical host into the virtual machine identifier and server IP comparison table.

Step S22: the method comprises the steps of distributing event notification files for a first virtual machine, and storing a first virtual machine identifier of the first virtual machine, memory use information of a target shared memory and file identifiers of the event notification files into a preset memory use information table, wherein the memory use information comprises a starting address and a memory size of the target shared memory, and the memory use information table is used for recovering resources when the first virtual machine is closed or is migrated from a first physical host.

In addition, an event notification file (eventfd) is allocated to the first virtual machine, and the first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and the file identifier of the event notification file are stored in a preset memory usage information table. The memory usage information includes a starting address and a memory size of the target shared memory. The memory usage information table is used for recovering resources of the shared memory when the first virtual machine is closed or is migrated from the first physical host to other physical hosts.

Step S23: and sending the memory use information of the target shared memory and the event notification file to PCIe equipment corresponding to the first virtual machine, wherein the memory use information is used for mapping the target shared memory to the PCIe memory in the PCIe equipment, and the event notification file is used for registering virtual interrupt for the first virtual machine.

And sending the memory usage information of the target shared memory and the event notification file to PCIe devices corresponding to the first virtual machine, where the memory usage information is used to map the target shared memory to PCIe memory in the PCIe devices, and the event notification file is used to register virtual interrupts for the first virtual machine.

And designating a Unix socket file when the first virtual machine is started, establishing connection with a part of the first physical host responsible for the allocation of the target shared memory, and transmitting data. After connection is established, the memory use information of the target shared memory, the event notification file and other information are sent to the PCIe equipment. The PCIe device maps the target shared memory to the PCIe memory according to the memory usage information of the target shared memory, and registers the event notification file eventfd as a virtual interrupt of the first virtual machine, which is used for virtual machine information synchronization. The virtual interrupt and the first virtual machine identifier can be stored in an event information table, so that corresponding virtual interrupt can be injected into the virtual machine according to the event information table, and an internal driver of the virtual machine can be triggered to receive a data packet.

In practical applications, the PCIe device provides configuration information and parameter information of the PCIe device for the first virtual machine in a register manner, for example, a memory size of the target shared memory, a first virtual machine identifier of the first virtual machine, an INTx interrupt, an interrupt number, a type, and the like. The PCIe device provides information for use in part by the device driver when initialized and for use in part by upper layer application connection setup, such as the first virtual machine identification, etc.

In order to realize efficient communication of a user layer, the device driver is constructed by using a UIO (Userspace I/O (input/output) technology) framework, wherein the device driver comprises a user mode driver and a kernel mode driver, the user mode driver encapsulates the use of the device driver by the user layer, and the kernel mode driver is responsible for registration and interrupt processing of the device driver. That is, the device driver may be implemented using the UIO framework, which is implemented in a separate manner: user mode and kernel mode. The kernel mode driver is responsible for registration and interrupt handling of the device driver. The user-state driver package uses the device driver.

Referring to fig. 3, after sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, the method further includes:

Step S31: and receiving a confirmation network data packet sent by the second physical host, wherein the confirmation network data packet is obtained by the second physical host by packaging the confirmation original data packet based on a first virtual machine identifier in the confirmation original data packet, and the confirmation original data packet is used for confirming whether the second virtual machine receives the original data packet to be transmitted or not.

Step S32: and analyzing the confirmation network data packet to obtain the confirmation original data packet.

Step S33: and transmitting the original acknowledgement data packet to the first virtual machine based on the first virtual machine identification in the original acknowledgement data packet.

After the first physical host sends the network data packet to be transmitted to the second physical host, the second virtual machine on the second physical host also returns a confirmation original data packet to the first virtual machine on the first physical host, so as to determine whether the second virtual machine receives the original data packet to be transmitted.

Specifically, the second virtual machine writes a confirmation original data packet into a PCIe memory of a PCIe device corresponding to the second virtual machine, the second physical host parses the confirmation original data packet to obtain a first virtual machine identifier in a packet header of the confirmation original data packet, finds an IP address of the first physical host corresponding to the first virtual machine identifier based on the first virtual machine identifier and a virtual machine identifier and server IP comparison table preset on the second physical host, encapsulates the confirmation original data packet into a confirmation network data packet by using the IP address of the first physical host, and sends the confirmation network data packet to the first physical host through a network, where the first physical host needs to receive the confirmation network data packet.

After receiving the confirmation network data packet, analyzing the confirmation network data packet to obtain the confirmation original data packet, and then sending the confirmation original data packet to the first virtual machine based on the first virtual machine identifier in the packet header of the confirmation original data packet, so that the first virtual machine determines whether the second virtual machine receives the original data packet to be transmitted. And if the original data packet to be transmitted is not received, the original data packet to be transmitted needs to be sent to the second virtual machine again.

Referring to fig. 4, the communication method between virtual machines further includes:

Step S41: and receiving the network data packets to be processed sent by other physical hosts except the first physical host.

Step S42: and analyzing the network data packet to be processed to obtain a raw data packet to be processed corresponding to the network data packet to be processed.

Step S43: and storing the original data packet to be processed into a shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the original data packet to be processed.

Step S44: and inserting a corresponding target virtual interrupt into the target virtual machine, wherein the target virtual interrupt is used for triggering a target device driver corresponding to the target virtual machine to receive the original data packet to be processed.

The first physical host also needs to receive a network data packet to be processed sent by other physical hosts except the first physical host, after receiving the network data packet to be processed, analyze the network data packet to be processed to obtain a raw data packet to be processed, store the raw data packet to be processed into a shared memory corresponding to a destination virtual machine identifier according to the destination virtual machine identifier in the raw data packet to be processed, and then insert a corresponding destination virtual interrupt into the destination virtual machine, wherein the destination virtual interrupt is used for triggering destination equipment corresponding to the destination virtual machine to drive and receive the raw data packet to be processed.

Specifically, after the original data packet to be processed is obtained, determining a shared memory corresponding to a target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed; and storing the original data packet to be processed into a write ring buffer queue in a shared memory corresponding to the destination virtual machine identifier. Determining a shared memory corresponding to a target virtual machine corresponding to the target virtual machine identifier according to the target virtual machine identifier in the original data packet to be processed and the memory usage information table, writing the original data packet to be processed into a write ring queue of the shared memory corresponding to the target virtual machine, determining virtual interrupt corresponding to the target virtual machine based on the event information table, inserting the virtual interrupt corresponding to the target virtual machine into the target virtual machine, triggering a device driver corresponding to the target virtual machine to receive the original data packet to be processed, wherein after the original data packet to be processed is read out from the write ring queue, the original data packet to be processed can be cached into a receiving cache queue corresponding to a virtual socket corresponding to the target virtual machine, so that the write ring queue can also receive the original data packet to be processed corresponding to other virtual sockets in the target virtual machine.

Referring to fig. 5, a flow chart of communication between virtual machines is shown. The method comprises the steps that a virtual machine sender APP (Application program) calls a lib library to create a virtual machine identifier comprising the virtual machine identifier, a port and a virtual socket vsocket of the port, wherein the virtual machine identifier is located by the virtual machine sender APP, the virtual socket 3225 is linked and written into an original data packet to be transmitted, a data packet forwarding module of a physical host A obtains the virtual machine identifier of the virtual machine receiver APP, an IP address of a physical host B of the receiver is obtained by inquiring a virtual machine identifier and a server IP comparison table, a data forwarding module which encapsulates the data packet to be transmitted and sends the data packet to be transmitted to the data forwarding module of the physical host B, the data forwarding module of the physical host B analyzes the packet head of the data packet to be transmitted, obtains the virtual machine identifier corresponding to the virtual machine identifier of the virtual machine receiver APP, inquires the event information table to obtain a virtual interrupt ioeventfd corresponding to the virtual machine identifier of the virtual machine receiver APP, writes the obtained original data packet to be transmitted into a shared ring-shaped memory corresponding to the virtual machine identifier of the virtual machine receiver APP, and confirms that the virtual machine is not connected with the virtual machine receiver APP, and the virtual machine is read out of the virtual machine is stopped, and the virtual machine APP is stopped from the virtual machine receiver APP, and the virtual machine is stopped from being read from the virtual machine receiver APP. And after that, when the virtual machine sender APP sends data to the virtual machine receiver APP again, virtual socket connection does not need to be established again, and when the virtual machine receiver APP processes the data packet in the write ring queue, the original data packet which cannot be processed in time can be discarded and retransmission of the virtual machine sender APP is required.

Referring to fig. 6, an embodiment of the present application discloses an inter-virtual machine communication device, which is applied to a first physical host running a first virtual machine, and includes:

The data reading module 11 is configured to read an original data packet to be transmitted in a target shared memory corresponding to the first virtual machine when it is monitored that a device driver corresponding to the first virtual machine performs a write operation on an I/O notification register in a PCIe device corresponding to the first virtual machine;

the IP address determining module 12 is configured to determine an IP address of the second physical host of the receiver based on the second virtual machine identifier in the original data packet to be transmitted and a preset virtual machine identifier and server IP comparison table;

And the data packet sending module 13 is configured to send the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, where the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier.

In some specific embodiments, the packet sending module 13 is configured to:

In some specific embodiments, the virtual machine communication apparatus further comprises:

The channel resource management module is used for determining the target shared memory from a local preset shared memory body when the first virtual machine is created on the first physical host, and distributing the target shared memory to the first virtual machine; and distributing an event notification file for the first virtual machine, and storing a first virtual machine identifier of the first virtual machine, memory use information of the target shared memory and a file identifier of the event notification file into a preset memory use information table.

In some specific embodiments, the channel resource management module is configured to:

the IP address receiving module is used for receiving a virtual machine identifier sent by the cloud platform management system to which the first virtual machine belongs and a physical host IP address corresponding to the virtual machine identifier;

and the comparison table creation module is used for constructing the comparison table of the virtual machine identifier and the server IP based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier.

In some specific embodiments, the look-up table creating module is configured to:

The data packet receiving module is used for receiving a confirmation network data packet sent by the second physical host, wherein the confirmation network data packet is obtained by encapsulating a confirmation original data packet by the second physical host based on a first virtual machine identifier in the confirmation original data packet, and the confirmation original data packet is used for confirming whether the second virtual machine receives the original data packet to be transmitted or not; analyzing the confirmation network data packet to obtain the confirmation original data packet; and transmitting the original acknowledgement data packet to the first virtual machine based on the first virtual machine identification in the original acknowledgement data packet.

In some specific embodiments, the data packet receiving module is further configured to:

In some specific embodiments, the data packet receiving module is configured to:

Referring to fig. 7, a diagram is configured for a method of inter-virtual machine process high-speed communication across physical machines in a cloud virtualization platform. The aforementioned first physical Host includes a Host operating system (Host OS) and a physical machine in the figure. In the communication device between virtual machines corresponding to the packet forwarding module in fig. 7, the data reading module 11, the IP address determining module 12, the packet transmitting module 13, the IP address receiving module, the lookup table creating module, and the packet receiving module are described. The channel resource management in the figure is the channel resource management module in the communication device between virtual machines. A Guest virtual machine (Guest VM) runs on Qemu, including the Guest operating system and the APP thereon. PCIe devices in the figures emulate and resource map, i.e., PCIe devices as previously described. The KVM extension in the figure is used to extend operations such as starting, migration, and closing of the virtual machine. The virtual machine management in the cloud platform management system is used for managing the virtual machines in the cloud platform, distributing virtual machine identifiers for the virtual machines, and sending the virtual machine identifiers and the IP addresses of the corresponding physical hosts to all the physical hosts in the cloud platform.

Reference is made to fig. 8, which is a schematic diagram of shared memory management. The channel management module in fig. 7 includes a shared memory bank management module and a memory usage information table, where the channel management module updates the memory usage information table according to a virtual machine update operation of KVM extension. And the channel management module distributes the shared memory in the preset shared memory body to the corresponding virtual machine, recovers the corresponding shared memory when the virtual machine is closed, and maps the shared memory corresponding to each virtual machine to the PCIe memory in the memory resource of the PCIe device.

Referring to FIG. 9, a PCIe device initialization schematic is shown. An event information table is maintained in a Host operating system (Host OS) of the physical Host, wherein the event information table comprises Virtual Machine Identifications (VMIDs) of the respective virtual machines and corresponding virtual interrupts (ioeventfd). And a virtual machine annular queue structure list including a write annular queue information structure (W-Rque information structure) and a read annular queue information structure (R-Rque information structure) corresponding to each virtual machine. The PCIe device comprises a register and a virtual interrupt (ioeventfd) registration module and memory resources of the device, wherein the memory resources are used for mapping corresponding shared memory, the device driver corresponding to the virtual machine comprises an interrupt application and registration module, and a write ring queue information structure (W-Rque information structure) and a read ring queue information structure (R-Rque information structure) are used for managing corresponding write ring queues and read ring queues.

Referring to fig. 10, a schematic structural diagram of a physical host 20 according to an embodiment of the present application is shown, where the physical host 20 may implement the steps of the virtual machine communication method disclosed in the foregoing embodiment.

In general, the physical host 20 in the present embodiment includes: a processor 21 and a memory 22.

Processor 21 may include one or more processing cores, such as a four-core processor, an eight-core processor, or the like, among others. The processor 21 may be implemented by at least one hardware selected from DSP (DIGITAL SIGNAL processing), FPGA (field-programmable array) GATE ARRAY, PLA (programmable logic array ). The processor 21 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (central processing unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a GPU (graphics processing unit, image processor) for taking care of rendering and drawing of images to be displayed by the display screen. In some embodiments, the processor 21 may include an AI (ARTIFICIAL INTELLIGENCE ) processor for processing computing operations related to machine learning.

Memory 22 may include one or more computer-readable storage media, which may be non-transitory. Memory 22 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 22 is at least used for storing a computer program 221, which, when loaded and executed by the processor 21, is capable of implementing the steps of the inter-virtual machine communication method disclosed in any of the foregoing embodiments.

In some embodiments, the physical host 20 may further include a display 23, an input/output interface 24, a communication interface 25, a sensor 26, a power supply 27, and a communication bus 28.

Those skilled in the art will appreciate that the architecture shown in fig. 10 is not limiting of the physical host 20 and may include more or fewer components than shown.

Further, the embodiment of the application also discloses a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the communication method between virtual machines disclosed in any of the previous embodiments.

The specific process of the communication method between virtual machines may refer to the corresponding content disclosed in the foregoing embodiment, and will not be described herein.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Finally, it is further noted that relational terms such as first and second are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a list of processes, methods, articles, or apparatus that comprises other elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above detailed description of the communication method, device, physical host and medium between virtual machines provided by the present application applies specific examples to illustrate the principles and embodiments of the present application, and the above description of the embodiments is only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.

Claims

1. A method for communication between virtual machines, applied to a first physical host running a first virtual machine, comprising:

Determining the IP address of a second physical host of the receiver based on the second virtual machine identification in the original data packet to be transmitted and a preset virtual machine identification and server IP comparison table; the virtual machine identification and server IP comparison table is a comparison table constructed based on the virtual machine identification and a physical host IP address corresponding to the virtual machine identification;

Sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier;

wherein the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host includes:

2. The method for communication between virtual machines according to claim 1, wherein, before the device driver corresponding to the first virtual machine is monitored to perform a write operation on an I/O notification register in a PCIe device corresponding to the first virtual machine, the method further comprises:

3. The method for communication between virtual machines according to claim 2, wherein in the process of determining the target shared memory from the local preset shared memory body, the method further comprises:

4. The method for communication between virtual machines according to claim 2, wherein after storing the first virtual machine identifier of the first virtual machine, the memory usage information of the target shared memory, and the file identifier of the event notification file in a preset memory usage information table, the method further comprises:

5. The method for communication between virtual machines according to claim 1, wherein, before the device driver corresponding to the first virtual machine is monitored to perform a write operation on an I/O notification register in a PCIe device corresponding to the first virtual machine, the method further comprises:

6. The method for communication between virtual machines according to claim 5, wherein after said constructing the virtual machine identifier and server IP lookup table based on the virtual machine identifier and the physical host IP address corresponding to the virtual machine identifier, further comprises:

7. The method according to claim 1, wherein after the sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, further comprising:

8. The method of communication between virtual machines according to any one of claims 1 to 7, further comprising:

9. The method for communicating between virtual machines according to claim 8, wherein storing the raw data packet to be processed in the shared memory corresponding to the destination virtual machine identifier according to the destination virtual machine identifier in the raw data packet to be processed includes:

10. An inter-virtual machine communication apparatus, for use with a first physical host running a first virtual machine, comprising:

The IP address determining module is used for determining the IP address of the second physical host of the receiver based on the second virtual machine identification in the original data packet to be transmitted and a preset virtual machine identification and server IP comparison table; the virtual machine identification and server IP comparison table is a comparison table constructed based on the virtual machine identification and a physical host IP address corresponding to the virtual machine identification;

The data packet sending module is used for sending the original data packet to be transmitted to the second physical host based on the IP address of the second physical host, wherein the original data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier;

Wherein, the data packet sending module includes:

The packaging unit is used for packaging the original data packet to be transmitted into a network data packet to be transmitted based on the IP address of the second physical host;

And the network data packet to be transmitted is used for being transmitted to a second virtual machine corresponding to the second virtual machine identifier after being analyzed, and the second virtual machine operates on the second physical host.

11. A physical host comprising:

A memory and a processor;

Wherein the memory is used for storing a computer program;

The processor for executing the computer program to implement the method of communication between virtual machines of any one of claims 1 to 9.

12. A computer readable storage medium for storing a computer program, wherein the computer program when executed by a processor implements the method of communication between virtual machines of any of claims 1 to 9.