CN115134859B - Intelligent congestion control method for diversified transmission requirements in highly dynamic networks - Google Patents

Abstract

The invention discloses an intelligent congestion control method and system facing diversified transmission demands in a high-dynamic network, wherein the development of the invention is mainly concentrated in an application layer and a transmission layer and comprises two modules: the invention relates to a channel capacity prediction module, which is used for predicting the capacity of a bottom network based on a long-short-term memory neural network to better adapt to a 5G network with high dynamic change of the channel capacity; the other is a reinforcement learning module, and the invention aims to design an intelligent congestion control algorithm based on reinforcement learning by combining specific bandwidth and time delay requirement values of application and preference of application among a plurality of requirement dimensions, thereby maximizing weighted requirement satisfaction rate and improving user experience.

Description

Intelligent congestion control method for diversified transmission requirements in highly dynamic networks

Technical Field

The present invention relates to the field of network communications, and in particular to an intelligent congestion control method and system for diversified transmission requirements in a highly dynamic network.

Background technique

In recent years, with the development of digital technology, the mobile network architecture has developed from 2G to 5G, the fifth generation of communication technology. Unlike 2G, which gave birth to data, 3G, which gave birth to data, and 4G, which developed data, 5G is a cross-era technology. It puts forward the concept of "user-centricity" and aims to provide users with a higher quality user experience. The computing power and transmission bandwidth of 5G networks have promoted the rapid development of various new applications such as Internet of Vehicles, high-definition video, and wireless medical care. Various applications have put forward diverse and clear transmission requirements in multiple dimensions such as throughput and latency. For example, autonomous driving applications require a latency of less than 10ms and a throughput of more than 1Gbps; 4K/8K high-definition video requires a latency of less than 10ms and a throughput of more than 100Mbps. In the process of data transmission, the network needs to meet the clear transmission performance requirements of diverse applications to ensure a good user experience.

In order to support the transmission needs of applications, 5G uses millimeter wave technology to provide greater channel capacity and lower latency. However, due to the use of higher frequencies, while achieving faster transmission rates and greater system capacity, millimeter wave technology also introduces serious propagation attenuation problems, causing the transmission quality to be easily affected by device movement and obstacles, that is, frequent switching between line-of-sight transmission and non-line-of-sight transmission (non-line-of-sight refers to the propagation path of the radio frequency that is partially blocked by obstacles, making it difficult for the wireless signal to pass). When switching between line-of-sight transmission and non-line-of-sight transmission, the channel capacity will change significantly, causing serious damage to the performance of the transmission protocol.

In order to give full play to the capabilities of 5G networks and provide high-quality transmission services for applications, the congestion control algorithm of the transport layer needs to consider both the clear transmission requirements of diversified applications and the highly dynamic underlying network environment. Since the sender cannot accurately perceive the channel capacity and its changes, the congestion control algorithm usually coordinates the amount of data injected into the network by adjusting the sending rate or congestion window. When the sending rate is too high or the congestion window is too large, the amount of data injected into the network will exceed the channel capacity, causing data packets to pile up in the intermediate forwarding devices, resulting in a large number of data packets queuing, and increasing end-to-end delay. When the sending rate is too low or the congestion window is too small, the throughput requirements of the application may not be met and the channel bandwidth capacity is not fully occupied. A good congestion control algorithm should achieve a good compromise between throughput and latency, making full use of the channel bandwidth while avoiding the latency caused by long queues.

Existing congestion control algorithms are mainly divided into traditional algorithms based on heuristics and intelligent algorithms based on machine learning. These algorithms are difficult to meet the clear transmission requirements of diversified applications in millimeter wave networks with highly dynamic capacity changes. Traditional congestion control algorithms usually simplify the network modeling based on the designer's experience, which cannot well reflect the actual situation of the network. Machine learning-based congestion control algorithms use more dimensional network status information to make congestion control decisions. Compared with traditional congestion control algorithms, they are more adaptable to dynamic network environments. By setting the objective function, a compromise effect of different weight preferences between throughput and delay can be obtained, providing differentiated transmission services for different applications to a certain extent. However, these two types of algorithms cannot perceive the clear multi-dimensional transmission demand values of applications during the design process, nor can they measure the completion of each demand indicator. In addition, the highly dynamic changes in the channel capacity of 5G networks will affect the convergence of existing learning-based machine learning congestion control algorithms.

Therefore, in order to make full use of the underlying network resources to bring better user experience to the upper-layer applications, the present invention intends to design an intelligent congestion control algorithm for diversified and clear transmission needs under a highly dynamic network. For the 5G network with highly dynamic underlying network channel capacity, intelligent congestion control is used to meet the clear demand values of upper-layer applications for transmission performance in multiple dimensions such as throughput and latency, thereby providing users with satisfactory transmission services.

Summary of the invention

In view of the above-mentioned deficiencies in the prior art, the present invention provides an intelligent congestion control method and control system for diversified transmission requirements in a highly dynamic network.

In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

An intelligent congestion control method for diversified transmission requirements in a highly dynamic network comprises the following steps:

S1. Sampling the millimeter wave channel capacity, and using the sampled channel capacity information to pre-train the channel capacity prediction model;

S2, predefined API interface, setting demand value and demand weight through the predefined API interface;

S3, statistics network status information and demand completion status at set time intervals, and deliver the statistics information to the congestion control algorithm. Among them, the channel capacity prediction module calculates the channel capacity prediction value based on historical channel capacity information;

S4, the reinforcement learning module performs reinforcement learning based on the channel capacity prediction value, statistical network status information and demand completion status, outputs the congestion window adjustment strategy and adjusts the strategy, and calculates the sending interval in the form of window value/average RTT;

S5. Repeat steps S3-S4 until the transmission is completed.

Furthermore, the demand value and demand weight in S2 are specifically:

The throughput requirement in Mbps and the delay requirement value in ms, as well as the delay requirement weight ω _d and the throughput requirement weight ω _b , and ω _b +ω _d =1.

Furthermore, the calculation method of the demand completion status in S3 is:

Where b ⁰ is the throughput requirement, d ⁰ is the delay requirement, I is the single-dimensional requirement satisfaction and

Furthermore, the S3 specifically includes the following steps:

S31, updating the statistical RTT, throughput, number of packet losses, and estimated channel capacity each time an ACK data packet is received;

S32. When the timer exceeds the time interval, the average throughput, average RTT, minimum RTT, delay gradient, packet loss rate, demand completion rate, maximum network bandwidth capacity, and reward value within the current time interval are calculated;

S33, delivering the data calculated in step S32 to the congestion control algorithm. The channel capacity module calculates the channel capacity prediction value.

Furthermore, the channel capacity prediction value in S33 is calculated as follows:

S331, adding the channel capacity information counted in the current time interval to a historical channel capacity queue, where the historical channel capacity queue always maintains a fixed length of historical channel capacity information;

S332: Input the information in the historical channel capacity queue into the trained network capacity prediction model, and output the channel capacity for the next time interval.

Furthermore, the channel capacity prediction module and the reinforcement learning module are arranged in the transmission layer, wherein:

The channel capacity prediction module takes the historical channel capacity as input and the predicted future channel capacity as output, and inputs it into the state space of the reinforcement learning module to assist the reinforcement learning module in making decisions;

The reinforcement learning module uses future channel capacity information, current network status information and transmission demand completion as inputs of the state space of the reinforcement learning module. In each decision time period, the learner fits the congestion control strategy according to the DDPG algorithm, uses the demand completion as the reward function, and outputs the congestion window adjustment strategy.

The present invention has the following beneficial effects:

1. Congestion control based on the clear transmission performance requirements and performance preference weights of diverse applications can maximize the weighted demand satisfaction rate, improve user experience, and ultimately improve performance.

2. Combining current network status information with predicted future channel capacity for congestion control can better adapt to 5G networks with highly dynamic changes in underlying network channel capacity.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is a schematic diagram of the framework of an intelligent congestion control system for diversified transmission requirements in a highly dynamic network.

FIG2 is a schematic diagram of intelligent congestion control based on capacity prediction and reinforcement learning according to an embodiment of the present invention.

FIG3 is a flow chart of an intelligent congestion control method for diversified transmission requirements in a highly dynamic network according to an embodiment of the present invention.

FIG4 is a schematic diagram showing how the demand satisfaction rate is improved in accordance with an embodiment of the present invention compared to a traditional algorithm.

Detailed ways

The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

An intelligent congestion control system for diversified transmission requirements in a highly dynamic network, as shown in FIG1, includes an application layer and a transmission layer, wherein the transmission layer includes a channel capacity prediction module and a reinforcement learning module. Specifically, as shown in FIG2,

The channel capacity prediction module takes the historical channel capacity as input and the predicted future channel capacity as output, and inputs it into the state space of the reinforcement learning module to assist reinforcement learning in making decisions. In this embodiment, the underlying network capacity is predicted based on the long short-term memory neural network to better adapt to the 5G network with highly dynamic changes in channel capacity.

The reinforcement learning module uses future channel capacity information, current network status information and transmission demand completion as inputs of the state space of the reinforcement learning module, and in each decision time period, the learner fits the congestion strategy according to the DDPG algorithm, uses the demand completion as a reward function, and outputs the congestion window adjustment strategy. In this embodiment, it is planned to design an intelligent congestion control algorithm based on reinforcement learning, combined with the specific bandwidth and delay demand values of the application and the preference of the application among multiple demand dimensions, so as to maximize the weighted demand satisfaction rate and improve the user experience.

The specific definition is as follows: application-defined throughput requirement b ⁰ , delay requirement d ⁰ , and throughput and delay preference weights ω _b and ω _d , and ω _b +ω _d = 1. _{b t} and d _t represent the throughput and delay at time t. The application requirement fulfillment at time t can be defined as in formula (1):

in It represents the satisfaction of a single dimension requirement. When the value is 1, it means that the requirement value of this dimension has been met. Otherwise, when it is not 1, it indicates the satisfaction ratio of the performance requirement. The reward function is shown in formula (2).

An intelligent congestion control method for diversified transmission requirements in a highly dynamic network, as shown in FIG3 , includes the following steps:

S1. Sampling the millimeter wave channel capacity, and using the sampled channel capacity information to pre-train the channel capacity prediction model;

Before the algorithm is used, the millimeter wave channel capacity is sampled, and the channel capacity prediction model (such as long short-term memory neural network) is pre-trained using the sampled channel capacity information within a period of time.

S2, predefined API interface, setting demand value and demand weight through the predefined API interface;

In this embodiment, the application sets specific demand values and demand weights through a predefined API interface, including the throughput demand in Mbps and the delay demand value in ms, as well as the delay demand preference ω _d and the throughput demand preference ω _b , where ω _b +ω _d = 1. If no algorithm is specified, the default value is used.

S3. Count the network status information and demand completion status at set time intervals, and deliver the counted information to the congestion control algorithm.

In this embodiment, the statistical network status information will be delivered to the congestion control algorithm, which includes a channel capacity prediction module and a reinforcement learning module. The statistical channel capacity information is used to predict the channel capacity, and the remaining information is input into the reinforcement learning module for reinforcement learning. Specifically, it includes the following steps:

S31, updating the statistical RTT, throughput, number of packet losses, and estimated channel capacity each time an ACK data packet is received;

S32. When the timer exceeds the time interval, the average throughput, average RTT, minimum RTT, delay gradient, packet loss rate, demand completion rate, maximum network bandwidth capacity, and reward value within the current time interval are calculated. The reward value calculation is shown in formula (2).

S33, delivering the data calculated in step S32 to the congestion control algorithm. The channel capacity prediction module calculates the channel capacity prediction value, and the specific prediction method is:

S331, adding the channel capacity information counted in the current time interval to a historical channel capacity queue, where the historical channel capacity queue always maintains a fixed length of historical channel capacity information;

S332: Input the information in the historical channel capacity queue into the trained network capacity prediction model, and output the channel capacity for the next time interval.

S4, the reinforcement learning module performs reinforcement learning based on the channel capacity prediction value, statistical network status information and demand completion status, outputs the congestion window adjustment strategy and adjusts the congestion window, and calculates the sending interval in the form of window value/average RTT;

S5. Repeat steps S3-S4 until the transmission is completed.

Based on the above scheme, the algorithm (QoSCC) proposed by the present invention improves the demand satisfaction rate compared with the traditional algorithm as shown in Figure 4. It can be seen that under different transmission demand settings, the actual transmission performance achieved by the present invention is different, but both meet the application requirements in terms of delay and throughput. Other algorithms can only achieve a fixed transmission effect, and under different transmission demand settings, the application requirements may not be met in terms of delay or throughput.

The present invention is described with reference to the flowchart and/or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims (4)

1. An intelligent congestion control method facing diversified transmission demands in a high dynamic network is characterized by comprising the following steps:

S1, sampling millimeter wave channel capacity, and pre-training a channel capacity prediction model by using sampled channel capacity information;

S2, predefining an API interface, and setting a requirement value and a requirement weight through the predefined API interface;

S3, counting the network state information and the demand completion condition at set time intervals,

The calculation mode of the requirement completion condition is as follows:

wherein, The degree of completion of the demand at time t,In order to meet the throughput requirements of the users,In order for the time delay requirement to be met,AndIndicating the throughput and the time delay at time t,AndFor throughput and delay preference weight values, I is one-dimensional demand satisfaction and；

Delivering the counted information to a congestion control algorithm, wherein the channel capacity prediction module calculates a channel capacity prediction value through historical channel capacity information, and specifically comprises the following steps of:

S31, updating the counted RTT, throughput, packet loss number and estimated channel capacity when each ACK data packet is received;

S32, when the timer exceeds the time interval, calculating average throughput, average RTT, minimum RTT, time delay gradient, packet loss rate, demand completion degree, maximum network capacity and rewarding value in the current time interval;

S33, delivering the data obtained by calculation in the step S32 to a congestion control algorithm, and calculating a channel capacity predicted value by a channel capacity prediction module;

S4, the reinforcement learning module performs reinforcement learning according to the channel capacity predicted value, the counted network state information and the demand completion condition, outputs a congestion window adjustment strategy and adjusts the congestion window, and simultaneously, according to the channel capacity predicted value, the counted network state information and the demand completion condition In the form of (a) calculates a transmission interval;

S5, repeating the steps S3-S4 until the transmission is completed.

2. The intelligent congestion control method for diversified transmission requirements in a high dynamic network according to claim 1, wherein the requirement value and the requirement weight in S2 are specifically:

throughput requirement in Mbps and latency requirement value in ms and latency requirement weight Sum throughput demand weightAnd (2) and。

3. The intelligent congestion control method for diversified transmission requirements in a high dynamic network according to claim 1, wherein the calculating manner of the channel capacity predicted value in S33 is as follows:

S331, adding channel capacity information counted by a current time interval into a historical channel capacity queue, wherein the historical channel capacity queue maintains a section of historical channel capacity information with a fixed length all the time;

s332, inputting the information in the historical channel capacity queue into a trained network capacity prediction model, and outputting the channel capacity of the next time interval.

4. The method for intelligent congestion control for diverse transmission demands in a highly dynamic network according to any one of claims 1-3, wherein the channel capacity prediction module and the reinforcement learning module are disposed in a transmission layer,

The channel capacity prediction module takes historical channel capacity as input, takes predicted future channel capacity as output, inputs the predicted future channel capacity into a state space of the reinforcement learning module, and assists the reinforcement learning module in making decisions;

The reinforcement learning module takes future channel capacity information, network state information at the current moment and transmission demand completion degree as input of a state space of the reinforcement learning module, and in each decision time period, the learner fits a congestion control strategy according to DDPG algorithm, uses the demand completion degree as a reward function, and outputs a congestion window adjustment strategy.