[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

CN114337893B - Accurate time synchronization method based on programmable data plane - Google Patents

Accurate time synchronization method based on programmable data plane Download PDF

Info

Publication number
CN114337893B
CN114337893B CN202111631275.1A CN202111631275A CN114337893B CN 114337893 B CN114337893 B CN 114337893B CN 202111631275 A CN202111631275 A CN 202111631275A CN 114337893 B CN114337893 B CN 114337893B
Authority
CN
China
Prior art keywords
time
switch
packet
synchronization
data packet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111631275.1A
Other languages
Chinese (zh)
Other versions
CN114337893A (en
Inventor
颜金尧
殷天航
王驰朝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Communication University of China
Original Assignee
Communication University of China
Filing date
Publication date
Application filed by Communication University of China filed Critical Communication University of China
Priority to CN202111631275.1A priority Critical patent/CN114337893B/en
Publication of CN114337893A publication Critical patent/CN114337893A/en
Application granted granted Critical
Publication of CN114337893B publication Critical patent/CN114337893B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Abstract

An accurate time synchronization method based on a programmable data plane relates to the technical field of time synchronization. The invention is in the application scenes of industrial automation, distributed multipoint synchronization system, ultra-high definition video and audio system broadcasting network, and the like, is very sensitive to the time synchronization precision in the network, which requires corresponding technology to enable the network to support higher time synchronization precision. The invention provides a time synchronization combination technology of a network card and a P4 switch based on a programmable data plane technology, which realizes nanosecond time synchronization and optimizes synchronization precision under different link conditions. In addition, the invention provides a brand new framework: the original network card receiving packet acceleration technology is combined with the data plane time synchronization technology, and the machine learning method is adopted to optimize the acquired synchronization error parameters, so that the synchronization error is reduced, the synchronization mode is optimized, and the most suitable accurate time synchronization mode under different scenes is selected.

Description

Accurate time synchronization method based on programmable data plane
Technical Field
The invention relates to the technical field of time synchronization, in particular to a method and a system for accurate time synchronization based on a programmable data plane. The invention is in the application scenes of industrial automation, distributed multipoint synchronization system, ultra-high definition video and audio system broadcasting network, and the like, is very sensitive to the time synchronization precision in the network, which requires corresponding technology to enable the network to support higher time synchronization precision. The traditional synchronization method cannot meet the requirements of high-precision synchronization scenes due to high actual deployment cost, low synchronization precision and the like. Meanwhile, the traditional network architecture is difficult to adapt to the requirement of the current fast-developing upper layer application on time synchronization because of the characteristics of poor sealing and openness. The proposal of the software defined network and the programmable technology provides a new idea for service scenes such as ultra-high definition video and audio playing networks and the like which need high-precision synchronization technology.
Background
DPTP data plane time synchronization protocol for synchronization between switches and supports synchronization accuracy on the nanosecond level. For the current development, the time synchronization protocol is generally applied to industrial switches, such as TSN (time sensitive network) switches, which are implemented based on IEEE 1588, and process the clock information at the control level.
PTP adopts master-slave clock synchronization mode, and time or frequency synchronization is realized between master clocks and slave clocks through messages such as interactive synchronization, state and time delay measurement. The PTP adopts a hardware time stamp, so that the precision and accuracy of the time stamp are higher, and the nanosecond synchronization precision can be realized generally, and the PTP is widely applied to high-precision time synchronization solutions of systems such as a communication transmission network, a mobile backhaul network, a smart grid, a high-speed railway and the like.
SDN software defined networking technologies monitor and manage multiple network facilities through OpenFlow protocols using a centralized controller through decoupling of the network facilities control plane and the data plane. In the software defined network architecture, the control plane is logically and uniformly controlled, and monitors the whole network link, and the control plane command is sent to the data layer of each device in the network by using the southbound OpenFlow protocol, so that the uniform management of the network device is enhanced by separating control and forwarding, and the method is beneficial to simplifying the deployment steps on each device.
Disclosure of Invention
The invention provides a network card and programmable data plane time synchronization combination technology based on a programmable data plane technology, which realizes nanosecond time synchronization and optimizes synchronization precision under different network conditions. In addition, the invention provides a brand new framework: the original network card receiving packet acceleration technology is combined with the data plane time synchronization technology, and the machine learning method is adopted to optimize the acquired synchronization error parameters, so that the synchronization error is reduced, the synchronization mode is optimized, and the most appropriate accurate synchronization time is selected under different network scenes, so that a better synchronization effect is achieved.
A method of accurate time synchronization based on a programmable data plane, comprising processing data packets at the programmable data plane pair; accelerating the data packet receiving and transmitting of the network card by adopting a DPDK+Lua script language frame; predicting the synchronous delay and the link delay by adopting a machine learning method, and optimizing the whole synchronous system;
When a request data packet is generated from a network switch, a request response time axis starts, and when a main switch receives the corresponding response data packet, two main-slave P4 switches are designed, the main switch sends a synchronous request data packet to the main switch, after receiving the data packet, the main switch writes the time stamp information of a data plane of the main switch into the synchronous request data packet, then generates the response data packet and sends the response data packet to the main switch; after receiving the response data packet from the switch, extracting the time stamp information carried in the data packet, wherein the time stamp information comprises various delay components involved in a request-response data packet model; in the data plane, the programmable data plane provides TRx, TIg, TEx and TTx four timestamp components, TRx representing the time at which the data packet entered the parser; TIg denotes the time when the packet enters the ingress pipeline; TEx represents the time when the packet entered the egress pipeline; TTx represents the time the data packet exits the inverse parser;
the processing of the data packet on the programmable data plane comprises the following steps:
Step 201, two switches are deployed, SW1 is a slave switch for clock calibration, SW2 is a master switch for reference time determination;
step 202, the master switch SW2 obtains a timestamp value TExt from an external clock source as a reference clock TRefsw of the slave switch SW1 for the current data plane, and simultaneously stores a timestamp TRespIg obtained inside the data plane SW2 and entering the port pipeline into a register, and records the current timestamp as Toffset (toffset= TRespIg);
In step 203, the x-bit counter is kept not cleared in the period of 2^x-1 seconds, a register is added to ensure that the counter is prevented from being cleared for a longer time, and when the counter is about to be cleared, the register records the time before the counter is cleared, and the time information is recorded as Tera;
Step 204, when the master switch SW2 receives a synchronization request packet, it reads the clock source timestamp value TRefsw obtained from the outside, the timestamp terra of the corresponding register of the time slot and the time offset variable Toffset of the incoming port pipeline;
step 205, while the master switch SW2 is outputting the port pipeline, it will read the current output port timestamp TRespEg;
In step 206, the current timestamp TNowsw after adding the reference time TRefsw is calculated using the following formula, and TNowsw is calculated from the timestamp t_offset of the ingress pipeline, and the current timestamp value is:
Step 207, since variable TNowsw uses the egress pipeline TRespEg time value to calculate inaccurately in order to avoid latency in the egress queue, step 206 is repeated to calculate TNowsw variable values using TRespEg each time a switch receives a request packet from another switch;
Step 208, the slave switch SW1 sends a request packet at time TReqTx; the request packet is received by the master switch SW2 at time TReqRx, and the clock source timestamp value TRefsw obtained from outside is read at the master switch SW2;
Step 209, in the data plane, the master switch SW2 matches the flow table by executing the action matching table, embeds TRefsw, TReqRx, and TRespEg time variables into the request packet, and the request packet is sent out by the master switch as a response packet at this time, and is retransmitted back to the slave switch SW1;
step 210, the slave switch SW1 receives the response data packet at time TRespRx, repeats step 206, and calculates the current time TNowsw of the master switch SW2;
step 211, processing the response packet from the switch SW1 at time TRespIg, where TRespIg is written into a register through a data plane register, denoted as Toffset;
Step 212, calculating the reference clock of the current slave switch, wherein the slave switch SW1 also needs to receive the accurate sending time of the data packet when the master switch is needed, so that the master switch SW2 will send a follow_up packet, and the data packet includes the time stamp of the response data packet accurately sent by the needed master switch, which is marked as TRespTx;
Step 213, after obtaining TRespTx variable values from the switch SW2, calculates the total delay as RespD by using the formula:
In step 214, the current time TNowsw of the master switch SW2, and the total delay RespD, the synchronization time of the slave switch SW1 can be accurately calculated by the following formula:
step 215, step 201 to step 214 are repeatedly executed.
Further, a DPDK+Lua script language frame is adopted to accelerate the network card data receiving and transmitting package, DPDK is adopted to accelerate the network card data receiving and transmitting package on hardware, and Lua script language is adopted to realize initialization and transmission of synchronous request package on software;
The method for accelerating the network card data receiving and transmitting package by adopting the DPDK+Lua script language frame comprises the following steps:
Step 301, installing a DPDK on a host, configuring a large page memory Hugepages, and loading a kernel module;
Step 302, defining a synchronous request packet by using a Lua script language, and defining a data packet sending rule;
Step 303, in the initialization stage, when no cross traffic exists in the link, the host will first send DPTP probe packets to the programmable switch, and the switch replies each timestamp TNowsw, TRespRx, TReqTx, TReqRx, TRespTx to the host; TNowsw is the current slave switch time, TRespRx is the time when the response packet arrives at the network card, TReqTx is the time when the host sends the detection packet, TReqRx is the time when the switch receives the detection packet, and TRespTx is the time when the response packet is sent out of the port;
Step 304, in the initialization phase, when the link rate r=0%, the link delay NICWIREDELAY at this time is calculated according to the following formula:
NicWireDelay=(TRespRx-TReqTx)-(TRespTx-TReqRx)
In step 305, in the synchronization stage, the reference time RespD of the host is calculated according to the following formula, TRespTx is the time when the switch sends out the response packet, TRespEg is the time when the response packet is pipelined out of the switch exit port:
RespD=OWD+(TRespTx-TRespEg)
Step 306, determining an uncomputed delay OWD, if the link rate R is about 0% during the synchronization phase, where owd= NICWIREDELAY/2, and once R is not 0%, NICWIREDELAY increases linearly, so that NICWIREDELAY calculated in the idle link during the initialization phase of step 303, denoted AVGNICWIREDELAY, is needed to calculate OWD in order to ensure that synchronization is accurate and that the clock on the host is synchronized with the switch; OWD is calculated according to the following formula:
step 307, calculating the reference time RespD of the host by using the formulas in steps 305 and 306, so as to achieve synchronization between the host and the programmable switch;
step 308, repeating steps 301 to 307.
Further, predicting and compensating the synchronous delay and the link delay by adopting a machine learning method, optimizing the whole synchronous system, counting the total delay and the unidirectional link delay between the switches in different network states under the condition of load and no load, predicting and compensating the total delay and the unidirectional link delay between the switches by adopting a cyclic neural network LSTM, counting the delay at the moment when the synchronization starts and taking the delay as an input layer, thereby predicting the current output delay, taking the value at the moment as the input layer, and iterating;
the method adopts a machine learning method to predict and compensate the synchronous delay and the link delay, and optimizes the whole synchronous network system specifically comprises the following steps:
step 401, according to step 213, calculating the total delay RespD between the master and slave switches;
Step 402, according to step 306, calculating an uncalculated delay OWD between the host and the switch;
step 403, counting total delay between switches in different network states, unidirectional link delay;
Step 404, after the whole network is synchronized, as the network has cross flow and various congestion and packet loss phenomena, respD and OWD can generate different amplitude changes, respD and OWD are taken as input x, put into a circulating neural network LSTM, and new RespD and OWD are predicted to be taken as output h;
step 405, predicting a new RespD, repeating step 214 to calculate the synchronization time of the slave switch SW 1;
Step 406, repeating step 305 to calculate the reference time of the host computer by using the predicted new OWD;
Step 407, repeating steps 401 to 405, and continuously updating the synchronization time of the slave switch and the host.
The invention realizes that the time stamp variable values of all components are embedded into the synchronous data packet in the data plane, and also realizes that the synchronous precision is within 25 nanoseconds when compared with the traditional time synchronous method under the condition of measuring a no-load link, and simultaneously simulates the condition of a complex link, and the synchronous precision is measured within 35 nanoseconds. When the actual physical topology is built, a p4c compiler is used for compiling the actual physical topology into json files to be configured in programmable network equipment, and a DPDK running environment is installed on a server-side network card, so that synchronization of the network card and a switch can be performed.
Drawings
Fig. 1: network card (server) and switch time synchronization technical architecture
Fig. 2: host.lua function call procedure
Fig. 3: p4 code corresponding analysis chart
Fig. 4: switch part table matching rule
Fig. 5: RNN network structure
Fig. 6: LSTM network structure
Fig. 7: synchronous system logic topology diagram
Fig. 8: synchronization information for 2P 4 switches
Fig. 9: execution host.lua code File results
Fig. 10: executing host.lua code file result at network card end
Detailed Description
The invention provides a test scheme of network card (server) and switch time synchronization technology, and provides a complete technical architecture diagram.
Network topology design
The invention designs a simple data center network topology to verify and test the proposed network card (server) and time synchronization technology, the test comprises 4 host computers (2 servers, two ports of each server network card are respectively regarded as 1 host computer), 2P 4 switches (1 physical P4 switch is virtualized to 2), and a specific network topology diagram is shown in figure 7.
The test experiment of the whole synchronization technology comprises a Barefoot Tofino exchanger which is connected with 2 servers, and the servers adopt Intel X520-DA2 network cards (supporting 10G/25G). The P4 physical switch uses the 132 ports and 168 ports of the 10G fiber connection switch to form a loop link for virtualizing one physical switch as 2 switches. And the server network card part, X520-DA2, supports the acquisition of the hardware time stamp of the network card, connects the 0 port of the network card with the 154 port of the P4 switch, and connects the 1 port of the network card with the 136 port of the P4 switch so as to realize the synchronous data packet transmitting and receiving link of the network card 1-P4 switch-network card 2. The software environment comprises an SDE (Software Development Environment ) version of the P4 switch of bf-SDE-8.9.1, a server operating system of Ubuntu-16.04-server-amd64 version and a DPDK version of 20.08.
Secondly, building network topology
(1) Programmable data plane end
In this section, P4 code is written to enable the virtualization of the P4 physical switch and synchronization between two P4 switches. The P4 compiling command is used for compiling P4 codes corresponding to the business logic, and the data plane forwarding logic of the data packet is loaded into the P4 switch. The master switch transmits a synchronization data packet to the master switch based on the reference clock source time information, and calculates a synchronization error with the master switch based on the received response data packet to correct its own clock. The P4 code is then compiled in the P4 switch with P4c and the compiled code is loaded into the switch. After the codes are compiled, corresponding data packet processing logic is realized only at the data layer, and then a flow table is downloaded to the switch through a control plane, wherein the logic comprises the port from which the data packet is sent and received, and the like.
As shown in fig. 8, the respective delay values after synchronization of 2P 4 switches and synchronization errors of2 switches can be seen. As can be seen from the figure, after the synchronous data packet arrives at the master switch, the data packet parsing process takes 62 ns, the processing and queue waiting at the ingress port pipeline of the master switch takes 350 ns, the processing and data packet inverse parsing at the egress port pipeline takes 320 ns, and the round trip RTT from the switch sending the synchronous request data packet to receiving the response data packet is 905 ns. Meanwhile, the absolute synchronization precision between the master and slave P4 switches is 6 nanoseconds.
(2) Network card end of server
After the P4 switch is synchronized, the network card sends the switch a synchronization request packet with the same packet header format as the switch is synchronized, and the host. As can be seen from the figure, EAL (Environment Abstraction Layer, environmental abstraction layer) in DPDK detects and initializes the server and the network card device. The 2 ports of the Intel X520-DA2 network card are taken over by the EAL, so that the Linux kernel can be bypassed for packet processing when sending and receiving the synchronous request packet. Meanwhile, 2 ports of the Intel X520-DA2 network card are configured into independent 2 Host, namely Device 0 and Device 1, corresponding mac is 90:E2:BA:27:A7:34 and 90:E2:BA:27:A7:35, and the network port link rate is 10000Mbit/s. The method mainly realizes the configuration of the network card, the transmission and the reception of the data packet and the acceleration of the processing of the data packet. Firstly, a DPDK technology is started and configured in a Linux system, wherein the DPDK technology comprises the steps of inserting an IGB_UIO module, distributing a large page of memory for a server, and binding a network card to a driver compatible with the DPDK. And then executing host.lua code to realize that the network card sends the data packets to the synchronized P4 switch at the rate of 2000 data packets per second. The network card itself supports the hardware timestamp, the method of reading the physical timestamp through the network card driving API is realized, the timestamp sent by the network card is embedded in the sent data packet, and finally the server network card and the switch begin to carry out synchronous test.
The host lua code is executed at the network card end, and as a result, as shown in fig. 10, each time variable value of the network card for the time synchronization formula can be obtained. Host2 (Dev 1) sends timesync a packet, command is set to respond to the packet command (0 x 3), each time variable value defined in the header of the synchronous packet can be obtained after receiving the response packet, and when command is a time stamp command for grabbing the sent packet (0 x 6), each time variable value can also be calculated. The response packet information returned by the switch can be obtained from the figure, for example, the time for the switch to process the network card request packet is 53 nanoseconds, and in the patent, the absolute synchronization error between the Host1 and the P4 switch (single hop) is 7 nanoseconds.
Third, invention integral frame
The integral frame of the present invention mainly comprises three parts: the programmable data plane processes the data packet, accelerates the network card to receive and transmit the data packet by adopting a DPDK+Lua script language frame, predicts the synchronous delay by adopting a machine learning method, predicts the link delay and optimizes the whole synchronous system.
Processing the data packet part in the programmable data plane, according to the principal and subordinate synchronization principle of the programmable data plane, realizing the packet header format of the self-defined synchronous data packet, the data plane obtaining time stamp, the logic code embedded in the synchronous data packet and the corresponding flow table under the switch by utilizing the control plane by writing P4 codes, thereby realizing the synchronization between two or more P4 switches. After the two P4 switches complete synchronization, at this time, the network card at the server end starts to send a synchronization request data packet to the switch based on a continuous data packet sending module realized by the DPDK and the Lua script language, and after the P4 switch receives the data packet with the same format, the P4 switch processes the data packet on the data plane through an action-matching pipeline, including writing in the own master clock of the switch after synchronization and a timestamp component of the data plane. After all time information of the P4 switch is written into the data packet, the switch sends out a response data packet through the matching-action pipeline and forwards the response data packet back to the server side. After receiving the response data packet, the server side directly gives the synchronous request data packet to the upper layer application by using a DPDK data packet acceleration technology and bypasses the kernel to analyze and calculate time information. Based on the above scheme, a request response model and an experimental whole frame diagram of a network card (server) and a switch are provided, as shown in fig. 1.
The method comprises the following steps of adopting a DPDK+Lua script language frame to accelerate the data packet receiving and transmitting of the network card, wherein the method comprises the following steps: the network card receives the packet acceleration and the software realizes the packet receiving and transmitting. In the acceleration of receiving packets by the network card, the problem of inaccurate delay measurement is caused by considering that the processing mode of the traditional data packet is the modes of CPU interruption, memory copy, context switching, memory management and the like. Therefore, a DPDK technology for accelerating data packet processing is adopted, so that the method can bypass a Linux kernel protocol stack, reduce additional time expenditure caused by memory copying, management and the like, and improve the synchronization precision. In the software implementation receiving and transmitting package, the Lua script language is adopted. The high-rate packet generator MoonGen developed based on the DPDK packet acceleration technique adopts the DPDK technique as the bottom layer. On the one hand, as a 'decorator' of the DPDK, moonGen can fully utilize the characteristic advantages of the DPDK technology and support the accelerated data packet processing; on the other hand, the whole MoonGen is completely controlled by the Lua script language, a user can develop a corresponding Lua script according to own business logic to realize receiving and transmitting the data packet, and the Lua language is used as a lightweight embedded script language, so that the method has the characteristics of high efficiency, portability, simplicity and light weight. In the module, according to the requirement that a synchronous request packet needs to be sent in a Host-Switch system, a definition and a receiving and transmitting module of a packet header realized based on a Lua script language are deployed in a network card.
The method comprises the steps of predicting the synchronization delay and the link delay by adopting a machine learning method, optimizing the whole synchronization system part, predicting and compensating the total delay and the unidirectional link delay between the switches by adopting a cyclic neural network LSTM, counting the delay at the moment when synchronization starts and taking the delay as an input layer so as to predict the current output delay, iterating the value at the moment as the input layer so as to predict the new link total delay RespD and the uncomputed delay OWD, recalculating the synchronization time between the switches and between the host and the switches, avoiding factors influencing the synchronization time such as congestion, packet loss and the like in the network after network synchronization, further improving the synchronization precision, and selecting the most suitable accurate synchronization time under different network scenes so as to achieve better synchronization effect.
Various operation principles and implementation methods of the invention
Programmable data plane processing data packet
The programmable data plane uses the P4 language. The P4 program mainly comprises Header, parser, match-Action Table and Control Flow. The P4 program was designed in three parts, header.p4, parser.p4 and Main.p4, respectively. Wherein the header of the data packet to be supported is declared in header.p4; instantiating the packet header in Parser.p4 and analyzing the data packet according to the sequence and the command in the data packet; and writing a time stamp of nanosecond level to the data layer of the switch in the main.p4 into the data packet, executing corresponding forwarding logic at the same time, and defining a control flow program of the execution sequence of each matched action table.
(1)Head.p4
The header of the packet that the network device may process is declared in the head.p4, and Ethernet protocol, UDP protocol, IPV4 protocol and timesync protocol are processed in this experiment.
The packet format declaration in Timesync protocol is the same as the timesync. Lua to ensure the normal communication of packets between the network card and the switch. Such as UDP protocol, IPV4 protocol, etc. are not expanded, after the timesync protocol is declared, it is necessary to define a Parser to parse the packet, and write the extracted related information into the instance of each protocol header for use in the following action-matching table.
(2)Parser.p4
Parser typically starts at PARSER START and ends with return ingress, and the header information of the packets is extracted sequentially. The header needs to be instantiated before parsing, and Parser parsing is shown in FIG. 3.
Firstly, the Ethernet is instantiated and the Ethernet field in the data packet is extracted, corresponding information is stored in an instantiation object, and the Ethernet field of the extracted Ethernet packet header is judged to determine the Parser flow direction of the next step, and in the patent, the flow direction is TIMESYNC HEADER and IPv4 protocol support respectively. IPv4 and timesync are then instantiated, extracting the various field information of timesync in the packet.
(3)Main.p4
In the main.p4 file, first two other p4 files of the include are needed, then a plurality of action-matching tables are defined for realizing the writing of the time stamp into the data packet in the data plane, including the action-matching tables for realizing the three-layer route forwarding function, and the register data structure is utilized for temporarily storing the time stamp value in the data packet for the internal time synchronization of the switch. Only three consecutive action-matching tables mac_forward, copy_packet and timesync _ inform _cp are selected here to introduce the code flow, as shown in fig. 4.
It can be seen from the figure that, first, a matching operation is performed on the table mac_forward, the destination mac address in the packet is matched with the flow table entry in the table, and if the matching is successful (hit), a set_egr action is performed. In the set_egr action, the output port of the metadata and the metadata output port inherent in the P4 switch ASIC are modified according to the matched entry information. The purpose of this action is to modify the output port of the data plane packet by means of a downstream table, and to control the forwarding of the packet from the designated port at the control plane.
And matching the table copy_packet, matching the value of mdata.command with the entry in the table, and if the matching is successful, executing a modification_packet action, wherein in the modification_packet action, the operation of modifying the data packet according to the flow table is not required, only the information of each field in the timesync data packet is copied into metadata mdata, and then the updated data in the metadata is written into the timesync data packet.
The third match is table timesync _ inform _cp, which also matches the value of mdata.command with the entry in the table, and if the match is successful (hit), timesync _flag_cp_ learn action is performed. In this action no modification of the packet content according to the flow table is required, as shown in fig. 3, for informing the control plane to send the Follow up packet.
Accelerating the data packet receiving and transmitting of the network card by adopting a DPDK+Lua script language frame
The DPDK module is adopted to mainly realize the transmission and the reception of the accelerated data packet, and by installing a corresponding DPDK version on a server, hugepages (large page memory) and a loading kernel module are required to be configured in a command line mode. The DPDK application program is operated in the user space to transmit and receive the data packet by utilizing a data plane database provided by the DPDK application program, and the processing process of the data packet by a Linux kernel protocol stack is bypassed. The Linux kernel sees the DPDK application (including its compilation, connection and loading modes) as a common user mode process. And after the DPDK program is started, only one main thread is provided, and then some sub threads are created and bound to a specified CPU core to run.
The Lua script language framework is mainly used for initializing and sending the synchronous request packet. Mainly comprising 2 files, a timesync. Lua and a host. Lua file. Wherein, in the timesync.lua, the header format and protocol command constants of the request packet are declared; and sending the data packet in the host.lua file, extracting a time stamp variable value in the data packet, and completing calculation of time delay.
(1) Define synchronization request packet timesync
The packet header protocol of the data packet to be processed by the network device is declared in the timesync.lua, the fields defined by the packet are used for storing the timestamp values, and the main fields in the packet header in the timesync.lua are declared as follows:
table 1: main field declaration in timesync.lua header
Meanwhile, a command flag needs to be set for the data packet, so that when the switch or the network card receives the data packet, the type of the data packet can be known, and different code strategies can be executed. In this patent, 7 packet command flags are set, type_req, type_res, type_delay_req, type_delay_res, type_CAPTURE_TX, type_ GENDELAY _req, and type_ GENREQ, respectively. Wherein TYPE_REQ represents that the current data packet is a request data packet, TYPE_RES represents that the current data packet is a response data packet, TYPE_DELAY_REQ represents that the current data packet is a request data packet carrying a time DELAY, TYPE_DELAY_RES represents that the current data packet is a response data packet carrying a time DELAY, TYPE_CAPTURE_TX represents that the current data packet is a data packet carrying a grabbing and transmitting time stamp, TYPE_ GENDELAY _REQ is a request data packet generating a time DELAY, TYPE_ GENREQ is a generated request data packet. By embedding different commands into the data packets in the timesync.lua file, when the P4 switch receives the synchronization request data packet, the switch adopts different data packet processing strategies.
Corresponding to defining the packet header format and the packet execution command, a method of acquiring time information in the packet and a method of adding a command to the packet are also defined, as shown in table 2. The protocol packet header required to be processed for the synchronization request data packet is declared to be completed, then the data packet is sent to the P4 switch through the DPDK technology of the bottom layer, meanwhile, the time field information is extracted for the received response data packet, and the synchronization error is calculated and used for correcting the clock synchronization of the network card of the server.
Table 2: method for obtaining time information in data packet and method for adding command to data packet
Method for adding data packet command Method for obtaining data packet time stamp variable value
setCommand(int) getMagic()
getCommand() setMagic(int)
getReference_ts_lo()
getReference_ts_hi()
getEraTs()
getMacTs()
getIgTs()
getDelta()
getEgTs()
Corresponding to defining the format of the data packet header and the command for executing the data packet, a method for acquiring time information in the data packet and a method for adding a command to the data packet are also defined. At this time, the protocol packet header required to be processed by the synchronization request packet is already declared to be completed, then the packet is sent to the P4 switch through the underlying DPDK technology, and meanwhile, the time field information is extracted from the received response packet, and the synchronization error is calculated and used for correcting the clock synchronization of the network card at the server. (2) Data packet transmitting file host.1ua
The time synchronization process of the server network card and the P4 switch is as follows: designing and initializing a synchronous request data packet at a network card end of a server, then sending the request data packet and recording a timestamp value TReqTx at the moment; then the synchronous request data packet arrives at the P4 switch, and the current time stamp TReqTx is recorded; the synchronized P4 switch would calculate the current TNowsw according to the formula and write the timestamp variable value into the packet at the data plane of the P4 switch: the timestamp TReqTx at this time is then recorded in response to the packet being sent out of the egress pipeline of the P4 switch. Meanwhile, in order to reduce the influence of waiting delay of the response data packet in the outbound queue, an additional Follow_up data packet is sent, and a time stamp which is actually sent by the response data packet and is stored in the data packet is TReqTx; when the response data packet arrives at the network card, the timestamp TReqRx at the moment is recorded, so that the network card device can calculate the synchronization error between the network card device and the switch and is used for correcting the clock of the server network card. In addition to the individual time stamps mentioned above, the switch also calculates the link packet rate R.
In the host.1ua file, the API of the bottom device module is called to configure the number of transmit queues and the number of receive queues in the network card, and at the same time, a EnableTimestamps () method is called to open a timestamp, and the physical timestamp is read from the network card. Initialization and data packet transmission are realized in the INITIATESERVER () function, time delay between the network card and the switch is calculated in the STARTTIMESYNC () function, and the time stamp of the network card is corrected, so that synchronization of the network card and the switch network is realized.
Thirdly, predicting the synchronous delay and the link delay by using a machine learning method, and optimizing the whole synchronous system
Time series model
Time series predictive analysis is the use of characteristics of time of an event over a period of time to predict characteristics of the event over a period of time in the future. The method is a relatively complex predictive modeling problem, is different from the prediction of a regression analysis model, a time sequence model depends on the sequence of occurrence of events, the result generated by an input model after the sequence of values of the same size is changed is different, and the characteristics of the time sequence model are utilized to optimize synchronous error data, so that the most suitable accurate time synchronization mode under different scenes is selected.
(II) RNN and LSTM models
The most common powerful tool for time series models is the Recurrent Neural Network (RNN). Compared with the characteristic of mutually independent calculation results of the common neural network, the calculation result of each hidden layer of the RNN is related to the current input and the previous hidden layer result. By this method, the result of the RNN calculation has the characteristic of memorizing the result several times before.
(1) RNN model
A typical RNN network architecture is shown in FIG. 5, with the right side being the architecture developed for ease of understanding memory during computation. Briefly, x is the input layer, o is the output layer, s is the hidden layer, and t refers to the calculation of the first time; v, W, U are weights, where st=f (u×xt+w×st-1) is calculated when calculating the hidden layer state of the t-th time, so as to achieve the purpose of hooking the current input result and the previous calculation.
(A) And predicting and compensating the single link delay by using the RNN:
When synchronization starts, the synchronization error of the single link at the moment is calculated and is used as an input layer, so that the currently output single link synchronization error is predicted, the numerical value at the moment is used as the input layer, iteration is carried out, the synchronization error adapting to the whole network structure is calculated, the whole network is synchronized in time again by using the synchronization error, and the accurate synchronization effect is improved under different scenes.
(B) Limitation of RNN:
Since the RNN model needs to hook the calculation of the current implicit state with the calculation of the previous n times if long-term memory is required, i.e., st=f (u×xt+w1×st-1+w2×st-2+ & wn×st-n), the calculation amount increases exponentially, resulting in a significant increase in the time for model training, so that the RNN model is generally directly used for long-term memory calculation.
(2) LSTM model
The LSTM (Long Short-Term Memory) model is a variation of the RNN and is characterized by the addition of valve nodes of layers outside the RNN structure. Valves are of the 3 classes: forget valve, input valve and output valve. These valves may be opened or closed to determine whether the result of the memory state of the model network (the state of the previous network) output at that layer reaches a threshold value to be added to the current calculation at that layer.
The valve node uses a sigmoid function to calculate the memory state of the network as input; multiplying the valve output with the calculation result of the current layer to be used as the input of the next layer if the output result reaches the threshold value; if the threshold is not reached, the output result is forgotten. The weights of each layer including valve nodes are updated during each model back propagation training process, as shown in fig. 6.
(A) Memory function of LSTM model:
the memorization function of the LSTM model is implemented by these valve nodes. When the valve is opened, the training results of the previous model are related to the current model calculation, and when the valve is closed, the previous calculation results do not influence the current calculation any more. Thus, by adjusting the valve opening and closing we can achieve the effect of the early sequence on the final result.
(B) Optimizing the synchronous time error by using an LSTM model:
The time delay conditions (loaded and unloaded) of each part under different network states are respectively measured, the error is used as the input of the upper layer, the time delay of the lower layer is calculated, multiple iterations are carried out, the time synchronization error under the current network state is calculated, and the time synchronization of the current network is calibrated by the error, so that the accurate synchronization effect is improved under different scenes, but if the network state or the network structure is changed at this time, the output result of the upper layer is forgotten by utilizing the characteristics of a forgetting valve, namely the output result of the upper layer has no influence on the later result, and the effect of updating the synchronization time error in the training process is achieved.
Thus, the basic design description of the present invention has been completed.

Claims (2)

1. A method of accurate time synchronization based on a programmable data plane, comprising processing data packets at the programmable data plane; accelerating the data packet receiving and transmitting of the network card by adopting a DPDK+Lua script language frame; predicting the synchronous delay and the link delay by adopting a machine learning method, and optimizing the whole synchronous system;
When a request data packet is generated from a network switch, a request response time axis starts, and when a main switch receives the corresponding response data packet, two main-slave P4 switches are designed, the main switch sends a synchronous request data packet to the main switch, after receiving the data packet, the main switch writes the time stamp information of a data plane of the main switch into the synchronous request data packet, then generates the response data packet and sends the response data packet to the main switch; after receiving the response data packet from the switch, extracting the time stamp information carried in the data packet, wherein the time stamp information comprises various delay components involved in a request-response data packet model;
The processing of data at the programmable data plane comprises the following steps:
Step 201, two switches are deployed, SW1 is a slave switch for clock calibration, SW2 is a master switch for reference time determination;
Step 202, the master switch SW2 obtains a timestamp value TExt from an external clock source as a reference clock TRefsw of the slave switch SW1 for the current data plane, and simultaneously stores a timestamp TRespIg obtained inside the data plane SW2 and entering the port pipeline into a register, and records that the current timestamp is Toffset and toffset= TRespIg;
In step 203, the x-bit counter is kept not cleared in the period of 2^x-1 seconds, a register is added to ensure that the counter is prevented from being cleared for a longer time, and when the counter is about to be cleared, the register records the time before the counter is cleared, and the time information is recorded as Tera;
Step 204, when the master switch SW2 receives a synchronization request packet, it reads the clock source timestamp value TRefsw obtained from the outside, the timestamp terra of the corresponding register of the time slot and the time offset variable Toffset of the incoming port pipeline;
step 205, while the master switch SW2 is outputting the port pipeline, it will read the current output port timestamp TRespEg;
In step 206, the current timestamp TNowsw after adding the reference time TRefsw is calculated using the following formula, and TNowsw is calculated from the timestamp Toffset of the ingress pipeline, and the current timestamp value is:
Step 207, since variable TNowsw uses the egress pipeline TRespEg time value to calculate inaccurately in order to avoid latency in the egress queue, step 206 is repeated to calculate TNowsw variable values using TRespEg each time a switch receives a request packet from another switch;
Step 208, the slave switch SW1 sends a request packet at time TReqTx; the request packet is received by the master switch SW2 at time TReqRx; the externally obtained clock source timestamp value TRefsw is read at the master switch SW2;
Step 209, in the data plane, the master switch SW2 matches the flow table by executing the action matching table, embeds TRefsw, TReqRx, and TRespEg time variables into the request packet, and the request packet is sent out by the master switch as a response packet at this time, and is retransmitted back to the slave switch SW1;
step 210, the slave switch SW1 receives the response data packet at time TRespRx, repeats step 206, and calculates the current time TNowsw of the master switch SW2;
step 211, processing the response packet from the switch SW1 at time TRespIg, where TRespIg is written into a register through a data plane register, denoted as Toffset;
Step 212, calculating the reference clock of the current slave switch, wherein the slave switch SW1 also needs to receive the accurate sending time of the data packet when the master switch is needed, so that the master switch SW2 will send a follow_up packet, and the data packet includes the time stamp of the response data packet accurately sent by the needed master switch, which is marked as TRespTx;
Step 213, after obtaining TRespTx variable values from the switch SW2, calculates the total delay as RespD by using the formula:
In step 214, the current time TNowsw of the master switch SW2, and the total delay RespD, the synchronization time of the slave switch SW1 can be accurately calculated by the following formula:
step 215, repeatedly executing step 201 to step 214;
Accelerating network card data receiving and transmitting packets by using a DPDK+Lua script language frame, accelerating network card data receiving and transmitting packets on hardware by using DPDK, and simultaneously initializing and transmitting synchronous request packets on software by using Lua script language;
The method for accelerating the network card data receiving and transmitting package by adopting the DPDK+Lua script language frame comprises the following steps:
Step 301, installing a DPDK on a host, configuring a large page memory Hugepages, and loading a kernel module;
Step 302, defining a synchronous request packet by using a Lua script language, and defining a data packet sending rule;
Step 303, in the initialization stage, when no cross traffic exists in the link, the host will first send DPTP probe packets to the programmable switch, and reply each timestamp TNowsw, TRespRx, TReqTx, TReqRx, TRespTx from the switch to the host; TNowsw is the current slave switch time, TRespRx is the time when the response packet arrives at the network card, TReqTx is the time when the host sends the detection packet, TReqRx is the time when the slave switch receives the detection packet, and TRespTx is the time when the response packet is sent out of the port;
Step 304, in the initialization phase, when the link rate r=0%, the link delay NICWIREDELAY at this time is calculated according to the following formula:
NicWireDelay=(TRespRx-TReqTx)-(TRespTx-TReqRx)
In step 305, in the synchronization stage, the reference time RespD of the host is calculated according to the following formula, TRespTx is the time when the switch sends out the response packet, TRespEg is the time when the response packet is pipelined out of the switch exit port:
RespD=OWD+(TRespTx-TRespEg)
Step 306, determining an uncomputed delay OWD, if the link rate R is about 0% during the synchronization phase, where owd= NICWIREDELAY/2, and once R is not 0%, NICWIREDELAY increases linearly, so that NICWIREDELAY calculated in the idle link during the initialization phase of step 303, denoted AVGNICWIREDELAY, is needed to calculate OWD in order to ensure that synchronization is accurate and that the clock on the host is synchronized with the switch; OWD is calculated according to the following formula:
step 307, calculating the reference time RespD of the host by using the formulas in steps 305 and 306, so as to achieve synchronization between the host and the programmable switch;
step 308, repeating steps 301 to 307.
2. The method according to claim 1, characterized in that: predicting and compensating the synchronous delay and the link delay by adopting a machine learning method, optimizing the whole synchronous system, counting the total delay and the unidirectional link delay between the switches under different network states with load and without load, predicting and compensating the total delay and the unidirectional link delay between the switches by adopting a cyclic neural network LSTM, counting the delay at the moment and taking the delay as an input layer when the synchronization starts, thereby predicting the current output delay, taking the value at the moment as the input layer, and iterating;
the method adopts a machine learning method to predict and compensate the synchronous delay and the link delay, and optimizes the whole synchronous network system specifically comprises the following steps:
step 401, according to step 213, calculating the total delay RespD between the master and slave switches;
Step 402, according to step 306, calculating an uncalculated delay OWD between the host and the switch;
step 403, counting total delay between switches in different network states, unidirectional link delay;
Step 404, after the whole network is synchronized, as the network has cross flow and various congestion and packet loss phenomena, respD and OWD can generate different amplitude changes, respD and OWD are taken as input x, put into a circulating neural network LSTM, and new RespD and OWD are predicted to be taken as output h;
step 405, predicting a new RespD, repeating step 214 to calculate the synchronization time of the slave switch SW 1;
Step 406, repeating step 305 to calculate the reference time of the host computer by using the predicted new OWD;
Step 407, repeating steps 401 to 405, and continuously updating the synchronization time of the slave switch and the host.
CN202111631275.1A 2021-12-28 Accurate time synchronization method based on programmable data plane Active CN114337893B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111631275.1A CN114337893B (en) 2021-12-28 Accurate time synchronization method based on programmable data plane

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111631275.1A CN114337893B (en) 2021-12-28 Accurate time synchronization method based on programmable data plane

Publications (2)

Publication Number Publication Date
CN114337893A CN114337893A (en) 2022-04-12
CN114337893B true CN114337893B (en) 2024-11-12

Family

ID=

Similar Documents

Publication Publication Date Title
US11012365B2 (en) Changing a time sensitive networking schedule implemented by a softswitch
US7020722B2 (en) Synchronization of distributed simulation nodes by keeping timestep schedulers in lockstep
CN110457747A (en) A kind of network control system synergy emulation method based on NS3 and MATLAB
US5081601A (en) System for combining independently clocked simulators
US20040158442A1 (en) Simulation method, apparatus, and computer program using hybrid model
EP4089565A1 (en) Synchronizing dynamic link budgets with fgpa based simulation waveform state machines
CN115118617B (en) Method, system and storage medium for measuring intention driving network
CN109951494A (en) Emulate data processing method, device, emulator and storage medium
EP4089564A1 (en) Modeling waveform physical layer and channel for simulation using custom hardware
CN114337893B (en) Accurate time synchronization method based on programmable data plane
Le et al. Timed-automata based schedulability analysis for distributed firm real-time systems: a case study
Geng et al. Model-based cosimulation for industrial wireless networks
Lauer et al. Analyzing end-to-end functional delays on an IMA platform
CN114090423A (en) Automatic control method for chip verification
CN114337893A (en) Precise time synchronization method based on programmable data plane
EP4089567A1 (en) High volume data logging from hardware
EP4092563A1 (en) Modeling multiple hardware routers in custom hardware
Basile et al. Residuals-based fault diagnosis of industrial automation systems using timed and untimed Interpreted Petri nets
Bergström Automatic generation of network configuration in simulated time sensitive networking (TSN) applications
CN117440417A (en) Network prediction system, method, electronic device and storage medium
Triantafyllidis et al. ProMARTES: Accurate network and computation delay prediction for component-based distributed systems
Tian et al. Hybrid system simulation of computer control applications over communication networks
Kuiper et al. Latency analysis of homogeneous synchronous dataflow graphs using timed automata
EP4089566A1 (en) Configuring custom hardware to operate in configurable operational modes using a scripting language
CN114978926B (en) Simulation method and equipment suitable for deterministic network protocol

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination
GR01 Patent grant