CN118524063B

CN118524063B - Tenant multi-network access method and system

Info

Publication number: CN118524063B
Application number: CN202410995209.XA
Authority: CN
Inventors: 康俊燕
Original assignee: Zhongyulian Cloud Computing Service Shanghai Co ltd
Current assignee: Zhongyulian Cloud Computing Service Shanghai Co ltd
Priority date: 2024-07-24
Filing date: 2024-07-24
Publication date: 2024-09-13
Anticipated expiration: 2044-07-24
Also published as: CN118524063A

Abstract

The invention relates to the field of cloud computing, in particular to a tenant multi-network access method and a tenant multi-network access system, if a sub-network used by Pod on the same node does not all start a DPDK mode, a OVN management instance, a first OVS instance and a second OVS instance are created; if the load of the core bound by the first OVS instance is higher than a threshold value or the flow corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-start DPDK mode, and placing the data packets to be processed corresponding to the selected subnet into a corresponding preparation queue; if the load of the core bound by the second OVS instance is higher than a threshold value, acquiring the average number of data packets received by the DMA of the DPDK from a receiving queue of each network card in a preset time period, and adjusting a subnet and a DMA polling mode from a DPDK mode to a DPDK mode not to be started according to the load of the core bound by the first OVS instance and the second OVS instance and the average number. The invention improves the load balancing capability of multi-network access of tenants.

Description

Tenant multi-network access method and system

Technical Field

The invention relates to the field of computer networks, in particular to a tenant multi-network access method and system.

Background

The multi-tenant architecture in a cloud computing environment enables different users or organizations to share the same physical infrastructure, each tenant has its independent data and applications, and ensuring the security and isolation of these resources in the shared environment is critical. Network isolation is an important means of achieving inter-tenant security isolation. For physical machines and virtual machines, network isolation can be achieved through technologies such as special network hardware, VLAN, virtual switch (vSwitch), software Defined Network (SDN) and the like, so that network traffic of different tenants is guaranteed to be independent and safe. Pod, however, by default, shares the same network namespace so that they can be accessed directly to each other, increasing the difficulty of quarantine, and Pod is highly dynamic and resilient, frequent in creation and destruction, requiring network quarantine quick response and adjustment. In actual service, many scenarios need Pod to support multiple network cards, such as optimizing network functions, flexible network configuration, high redundancy, and the like.

Multus CNI allows one Pod to be connected to a plurality of networks, kube-OVN provides rich network functions, network isolation can be achieved by adopting a combined scheme of Multus CNI and Kube-OVN, and subnet management, load balancing, ACL, network policy and the like can be achieved, but configuring a plurality of network interfaces for each Pod increases resource overhead, including more network bandwidth, consumption of a CPU and memory and the like, which can have a larger influence on a large cloud computing platform, especially when tenants are many. It is of great importance how to reduce the performance overhead when Multus CNI and Kube-OVN are used for network isolation.

Disclosure of Invention

In order to solve the above problems, the present invention provides a tenant multi-network access method, which includes the following steps:

Installing Multus-CNI and Kube-OVN plug-ins in the cloud platform, creating a plurality of subnets using Kube-OVN and configuring at least one subnet for Pod using Multus-CNI; if the subnets used by Pod on the same node do not all start the DPDK mode, creating a OVN management instance, a first OVS instance and a second OVS instance;

OVN the management instance adds a plurality of preparation queues after receiving queues corresponding to the second OVS instance, wherein the number of the preparation queues is the same as the number of subnets which do not start a DPDK mode; if the load of the core bound by the first OVS instance is higher than a threshold value or the flow corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-start DPDK mode, and placing the data packets to be processed corresponding to the selected subnet into a corresponding preparation queue;

If the load of the core bound by the second OVS instance is higher than a threshold value, acquiring the average number of data packets received by the DMA of the DPDK from a receiving queue of each network card in a preset time period, and adjusting a subnet and a DMA polling mode from a DPDK mode to a DPDK mode not to be started according to the load of the core bound by the first OVS instance and the second OVS instance and the average number.

Preferably, if the load of the core bound by the first OVS instance is higher than a threshold value or the flow corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-turned-on DPDK mode, specifically:

if the load of the core bound by the first OVS instance is higher than a threshold value, acquiring the interruption frequency of each sub-network which does not start the DPDK mode, and selecting the sub-network with the largest interruption frequency;

and if the flow corresponding to the subnet is larger than a preset value, taking the subnet as a selected subnet.

Preferably, after the data packets to be processed corresponding to the selected subnet are placed in the corresponding preparation queues, the method further includes:

and OVN, the management example freezes the selected subnet in the first OVS example, sets the flag bit of the preparation queue to be readable, and the second OVS example adds the preparation queue into the DMA poll according to the flag bit in the preparation queue.

Preferably, the load of the core bound according to the first OVS instance and the second OVS instance, the average adjustment changes from a subnet with DPDK mode turned on to a subnet without DPDK mode turned off, and a DMA polling mode, specifically:

Calculating a load difference value between a load of a core bound by the second OVS instance and a load of the core bound by the first OVS instance, and a difference value between the load of the core bound by the second OVS instance and the threshold, and determining and adjusting a subnet which is changed from a DPDK mode to a non-DPDK mode according to the load difference value, the difference value and the average value; then, a DMA polling scheme is determined.

Preferably, the determining, according to the load difference value, the difference value and the average value, adjusts a subnet that is changed from the on DPDK mode to the off DPDK mode, specifically:

And pre-establishing a corresponding table of the weighted result and the average value, calculating according to the load difference value and the difference value by a weighted mode to obtain the weighted result, determining the average value corresponding to the corresponding table according to the weighted result, selecting at least one subnet closest to the average value, and adjusting the subnet from a DPDK (digital pre-distortion) mode to a DPDK (digital pre-distortion) mode.

Preferably, the determining a DMA polling mode specifically includes:

And if the difference between the load of the core bound by the second OVS example after adjustment and the threshold value is within a preset range, for starting the DPDK mode sub-network, determining a DMA polling interval corresponding to the sub-network according to the average number of the starting DPDK mode sub-network.

In addition, the invention also provides a tenant multi-network access system, which comprises the following modules:

a subnet instance creation module for installing Multus-CNI and Kube-OVN plug-ins in the cloud platform, creating a plurality of subnets using Kube-OVN and configuring at least one subnet for Pod using Multus-CNI; if the subnets used by Pod on the same node do not all start the DPDK mode, creating a OVN management instance, a first OVS instance and a second OVS instance;

a first adjusting module, configured to OVN the management instance to add a plurality of preparation queues after the receiving queues corresponding to the second OVS instance, where the number of the preparation queues is the same as the number of subnets in which the DPDK mode is not started; if the load of the core bound by the first OVS instance is higher than a threshold value or the flow corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-start DPDK mode, and placing the data packets to be processed corresponding to the selected subnet into a corresponding preparation queue;

And the second adjusting module is used for acquiring the average number of data packets received by the DMA of the DPDK from the receiving queue of each network card in a preset time period if the load of the core bound by the second OVS instance is higher than a threshold value, and adjusting the subnet and the DMA polling mode from the DPDK starting mode to the DPDK non-starting mode according to the load of the core bound by the first OVS instance and the second OVS instance.

Preferably, the determining a DMA polling mode specifically includes:

Furthermore, the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method as described above.

Many tenants exist on cloud computing, and resources of a system are limited, so that reasonable utilization of the resources is particularly important. Based on the method, the device and the system for adjusting the multi-network access of the tenants adjust the utilization of resources in the multi-network access of the tenants, specifically, whether a DPDK mode is started for a subnet in use or not is judged, the subnet is allocated to a corresponding OVS instance, and the mode of the subnet and the corresponding OVS instance are adjusted according to the load condition of a core bound by the OVS instance and the data traffic condition of the subnet. The load of the core of the OVS instance is dynamically adjusted, so that the high availability, performance and reliability of cloud computing are guaranteed, and the user experience and the overall efficiency of the system are improved.

Drawings

FIG. 1 is a flow chart of a first embodiment;

fig. 2 is a schematic diagram of the relationship between Pod and subnet and OVS instance;

fig. 3 is a structural diagram of the second embodiment.

Detailed Description

In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In the present invention, if the collection of personal privacy related data, such as face, mobile phone usage information, etc., personal permission is obtained in advance, including but not limited to verbal reminding, poster posting, mobile phone reminding, etc.; if the place which conflicts with laws and regulations is involved, the method is produced or used within the allowable range of laws and regulations.

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which are obtained by persons of ordinary skill in the art without undue burden on the person of ordinary skill in the art based on the embodiments of the present invention, are within the scope of the present invention.

In a first embodiment, the present invention provides a tenant multi-network access method, as shown in fig. 1, the method includes the following steps:

s1, installing Multus-CNI and Kube-OVN plug-ins in a cloud platform, creating a plurality of subnets by using Kube-OVN and configuring at least one subnet for Pod by using Multus-CNI; if the subnets used by Pod on the same node do not all start the DPDK mode, creating a OVN management instance, a first OVS instance and a second OVS instance;

Multus-CNI is a Kubernetes Container Network Interface (CNI) plug-in, pod can be enabled to access to a plurality of network interfaces through Multus-CNI, kube-OVN is a Kubernetes network plug-in constructed based on OVN (Open Virtual Network), and network functions in the Kubernetes cluster are realized mainly through OVN and OVS (Open vSwitch). When the Multus-CNI and Kube-OVN are used for realizing multi-network access, multus-CNI is used as an element plug-in to call the principal subnet, and Kube-OVN realizes the function of the principal subnet.

When implementing tenant multi-network access, first, multus-CNI and Kube-OVN plug-ins are installed in a cloud platform such as Kubernetes (K8S), then Kube-OVN are used to create multiple subnets, and Multus-CNI is used to configure subnets for Pod, as shown in FIG. 2, each Pod is configured with at least one subnet, the subnet connected with each Pod in solid line in FIG. 2 is the subnet in use, and the subnet connected with each Pod in dashed line is the additional subnet or standby subnet of the Pod. The subnet created by Kube-OVN has two data packet processing modes, namely a kernel path mode and a DPDK mode, wherein the kernel path mode needs to process data packets in a kernel mode and needs to be continuously switched between a user mode and the kernel mode, but the realization is simpler due to the adoption of a kernel module; the DPDK mode is used for processing the data packet in a user mode instead of a kernel mode, so that the switching of the system between the kernel mode and the user mode is reduced, the processing speed is high, flexible programming can be realized, and more complex functions can be realized.

And judging whether the DPDK modes are all started or not for the subnets which are used by the Pod on the same node, if not, namely, the subnets which are used by the Pod on the same node are started, wherein one part of subnets are started in the DPDK mode, and the other part of subnets are not started in the DPDK mode, and creating a OVN management instance, a first OVS instance and a second OVS instance. The node refers to a server or a host in cloud computing. OVN (Open Virtual Network) the management instance, also referred to as OVN instance, is used to provide related functions of the virtual network, such as load balancing, etc., and OVN components include north databases, south databases, OVN controllers, etc. The method mainly comprises the steps of determining the flow direction of a data packet and realizing the related function of a virtual network, adopting a first OVS example to process the data packet for a subnet which is not started in a DPDK mode, and adopting a second OVS example to process the data packet for the subnet which is started in the DPDK mode. The first OVS instance is configured to process a packet of a subnet that does not have a DPDK mode on, and the second OVS instance is configured to process a packet of a subnet that has a DPDK mode on.

S2, OVN management examples add a plurality of preparation queues after receiving queues corresponding to the second OVS examples, wherein the number of the preparation queues is the same as the number of subnets which do not start a DPDK mode; if the load of the core bound by the first OVS instance is higher than a threshold value or the flow corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-start DPDK mode, and placing the data packets to be processed corresponding to the selected subnet into a corresponding preparation queue;

In order to achieve load balancing, in the invention, a plurality of preparation queues are added in a second VOS example in advance, the number of the preparation queues is the same as the number of the subnets of the non-started DPDK mode, and a relation between the preparation queues and the subnets of the non-started DPDK mode is established in advance so that the preparation queues and the subnets of the non-started DPDK mode are in one-to-one correspondence. When the load of the core bound by the first OVS instance is higher than a threshold value or the flow of the subnet which is not started in the DPDK mode is higher than a preset value, selecting the subnet from the subnets which are not started in the DPDK mode, and placing the data packets to be processed corresponding to the selected subnet into a corresponding preparation queue. In a more specific embodiment, the subnet data without the DPDK mode is directly forwarded or mirrored through the port and stored in the corresponding preparation queue. In another more specific embodiment, a plurality of transmission queues are added after the transmission queues corresponding to the second OVS instance, and the number of the transmission queues is the same as the number of the preparation queues and corresponds to one, so that the second OVS instance can receive and transmit the data packet of the processed selected subnet, where the processing refers to performing various network functions on the data packet. The data packet of the first OVS instance is transferred to the second OVS instance for processing, so that on one hand, the load of the core where the first OVS instance is located can be reduced, and on the other hand, the processing speed can be increased for the sub-network of the large data flow.

In a more specific embodiment, if the load of the core bound by the first OVS instance is higher than a threshold value or the traffic corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the non-turned-on DPDK mode, specifically:

The core path mode, namely the sub-network which does not start the DPDK mode, wakes up the core to process the data packet in an interrupt processing mode, that is to say, when the data packet exists, the interrupt program triggers the interrupt, the data packet is copied to a buffer zone in the core, and further the processing of the data packet by the core is realized, however, the frequent interrupt causes the waste of system resources, particularly a CPU, and if the load of the core bound by the first OVS example is higher than a threshold value, the sub-network with the largest interrupt frequency of the sub-network which does not start the DPDK mode is taken as a selection target. In a more specific embodiment, the interrupt frequency is calculated by the number of kernel interrupts triggered by a subnet due to arrival of a data packet within a preset time; the preset time is 2min or 5min.

When the flow of the sub-network which does not start the DPDK mode is larger than a preset value, more data packets need to be processed, but the processing speed of the kernel path mode is slower, setting a preset value for the situation, and taking the sub-network as a selection target when the flow corresponding to the sub-network is larger than the preset value.

After the selection target is determined, the data to be processed corresponding to the selection target is placed into a corresponding preparation queue, then OVN management instance freezes the selected subnet in the first OVS instance, the flag bit of the preparation queue is set to be readable, and the second OVS instance adds the preparation queue into DMA polling according to the flag bit in the preparation queue.

The first OVS instance no longer processes the data packet of the selected subnet, at this time, the flag bit of the preparation queue corresponding to the selected subnet is set to be readable, so that the second OVS instance can read the data packet from the preparation queue and process it. The sub-network of the DPDK mode is turned on by adopting a polling mode, namely, the sub-network reads data packets from a receiving queue through the polling mode and processes the data packets, which is different from the core path mode.

And S3, if the load of the core bound by the second OVS instance is higher than a threshold value, acquiring the average number of data packets received by the DMA of the DPDK from a receiving queue of each network card in a preset time period, and adjusting a subnet and a DMA polling mode from a DPDK mode to a DPDK mode which is not started according to the load of the core bound by the first OVS instance and the second OVS instance and the average number.

When the load of the core bound by the second OVS instance is higher than a threshold value, acquiring an average number of data packets received by the DPDK from each receiving queue through a DMA (Direct Memory Access) polling mode in a preset time period, then adjusting a subnet converted from a DPDK on mode to a DPDK off mode according to the load of the core bound by the first OVS instance and the second OVS instance and the average data, and further adjusting the polling mode of the DMA after the transition.

In a specific embodiment, the load of the core bound according to the first OVS instance and the second OVS instance, the average adjustment changes from the subnet with DPDK mode turned on to the subnet without DPDK mode, and the DMA polling mode are specifically:

For example, the load of the core bound by the second OVS instance is 95%, the load of the core bound by the first OVS instance is 20%, the threshold is 90%, the load difference between the core where the second OVS instance and the first OVS instance are located is 75%, and the difference between the second OVS instance and the threshold is 5%. If the second OVS instance corresponds to 3 subnets, the average numbers corresponding per second are 1000, 2000 and 3000, respectively. From these data, the sub-network to be adjusted is determined.

In another embodiment, the determining, according to the load difference value, the difference value and the average value, to adjust the subnet that changes from the on DPDK mode to the off DPDK mode is specifically:

And carrying out weighted calculation on the load difference value and the difference value, wherein in one possible weighted calculation mode, the weights of the load difference value and the difference value are respectively set to be 0.5, and the weighted result is as follows: 0.4. according to the pre-established correspondence table, assuming that 0.4 corresponds to 1800, the mode of the sub-network with the average number of 2000 per second is converted to the non-turning on DPDK.

Polling also aggravates the load of the CPU core, and after adjusting the subnet mode, the DMA polling mode is further determined, specifically:

For example, the difference between the load of the core bound by the second OVS instance after adjustment and the threshold is 3%, if the preset range is 5%, for the subnet corresponding to the second OVS instance, determining the DMA polling interval corresponding to the subnet according to the average number of the remaining subnets. In a more specific embodiment, the average value and the average value of the subnets of the transition mode are determined, if the average value is greater than the average value of the subnets of the transition occurrence, the DMA polling interval is kept unchanged, and if the average value is less than the average value of the subnets of the transition occurrence, the DMA polling interval is doubled. In another embodiment, the determining, according to the average value of the remaining subnets, the DMA polling interval corresponding to each subnet is specifically: calculating an average value of the averages of the remaining subnets, if the average value is greater than or equal to the average value, keeping the polling interval unchanged, and if the average value is less than the average value, doubling the polling interval. Therefore, the time of the polling process can be shortened, all queues are prevented from being polled each time, and the polling efficiency is improved. In another embodiment, when the core load of the OVS instance falls below a threshold or the traffic of the subnet is less than a preset value, the adjusted subnet is restored. In order to avoid the subnet from oscillating between the first OVS instance and the second OVS instance, a time interval is set, and when the adjustment is completed, the adjusted subnet is not adjusted again in the time interval.

In a second embodiment, the present invention further provides a tenant multi-network access system, as shown in fig. 3, where the system includes the following modules:

Preferably, the determining a DMA polling mode specifically includes:

In a third embodiment, the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the method according to the first embodiment.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by adding necessary general purpose hardware platforms, or may be implemented by a combination of hardware and software. Based on such understanding, the foregoing aspects, in essence and portions contributing to the art, may be embodied in the form of a computer program product, which may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, but not limiting the same, and other embodiments may be adopted; although the invention has been described in detail with reference to the foregoing embodiments, it will be appreciated by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A tenant multi-network access method, the method comprising the steps of:

2. The method of claim 1, wherein if the load of the core bound by the first OVS instance is higher than a threshold value or the traffic corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the unopened DPDK mode, specifically:

3. The method of claim 1, wherein after placing the data packets to be processed corresponding to the selected subnet in the corresponding ready queue, further comprising:

4. The method of claim 1, wherein the load of the core bound according to the first OVS instance and the second OVS instance, the average adjustment changes from a subnet with DPDK mode turned on to a subnet without DPDK mode, and the DMA polling mode are specifically:

5. The method according to claim 4, wherein the determining to adjust the subnet to switch from the on DPDK mode to the off DPDK mode based on the load difference value, the difference value and the average value is specifically:

6. The method of claim 4, wherein the determining a DMA polling mode is specifically:

7. A tenant multi-network access system, the system comprising the following modules:

8. The system of claim 7, wherein if the load of the core bound by the first OVS instance is higher than a threshold value or the traffic corresponding to the subnet is greater than a preset value, selecting the subnet from the subnets in the unopened DPDK mode, specifically:

9. The system of claim 7, wherein after placing the corresponding pending data packets for the selected subnet in the corresponding ready queue, further comprising:

10. A computer storage device having stored thereon a computer program which, when executed by a processor, implements the method according to any of claims 1-6.