US20140029604A1 - Communicating time information in a network to enable synchronization - Google Patents
Communicating time information in a network to enable synchronization Download PDFInfo
- Publication number
- US20140029604A1 US20140029604A1 US14/043,013 US201314043013A US2014029604A1 US 20140029604 A1 US20140029604 A1 US 20140029604A1 US 201314043013 A US201314043013 A US 201314043013A US 2014029604 A1 US2014029604 A1 US 2014029604A1
- Authority
- US
- United States
- Prior art keywords
- node
- timing packet
- timing
- delay
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000012512 characterization method Methods 0.000 claims description 41
- 230000001934 delay Effects 0.000 claims description 28
- 238000010295 mobile communication Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 6
- 230000001186 cumulative effect Effects 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 abstract description 4
- 230000001360 synchronised effect Effects 0.000 description 8
- 230000004044 response Effects 0.000 description 6
- 230000015654 memory Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000005259 measurement Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 229910052792 caesium Inorganic materials 0.000 description 1
- TVFDJXOCXUVLDH-UHFFFAOYSA-N caesium atom Chemical compound [Cs] TVFDJXOCXUVLDH-UHFFFAOYSA-N 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 229910052701 rubidium Inorganic materials 0.000 description 1
- IGLNJRXAVVLDKE-UHFFFAOYSA-N rubidium atom Chemical compound [Rb] IGLNJRXAVVLDKE-UHFFFAOYSA-N 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0673—Clock or time synchronisation among packet nodes using intermediate nodes, e.g. modification of a received timestamp before further transmission to the next packet node, e.g. including internal delay time or residence time into the packet
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W56/00—Synchronisation arrangements
- H04W56/001—Synchronization between nodes
- H04W56/0015—Synchronization between nodes one node acting as a reference for the others
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/04—Generating or distributing clock signals or signals derived directly therefrom
- G06F1/12—Synchronisation of different clock signals provided by a plurality of clock generators
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
- H04J3/0664—Clock or time synchronisation among packet nodes using timestamps unidirectional timestamps
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04J—MULTIPLEX COMMUNICATION
- H04J3/00—Time-division multiplex systems
- H04J3/02—Details
- H04J3/06—Synchronising arrangements
- H04J3/0635—Clock or time synchronisation in a network
- H04J3/0638—Clock or time synchronisation among nodes; Internode synchronisation
- H04J3/0658—Clock or time synchronisation among packet nodes
- H04J3/0661—Clock or time synchronisation among packet nodes using timestamps
- H04J3/0667—Bidirectional timestamps, e.g. NTP or PTP for compensation of clock drift and for compensation of propagation delays
Definitions
- the invention relates generally to communicating time information across nodes in a network to enable synchronization of one or more of the nodes.
- nodes within a network have to be time-synchronized.
- a wireless or mobile communications network such as a CDMA (code division multiple access) 2000 and WiMAX (World Interoperability for Microwave Access)
- nodes within a mobile communications network have used global position system (GPS) receivers to maintain time synchronization.
- GPS global position system
- GPS receivers or other high accuracy time reference sources provided in each node add to the hardware and installation cost of network equipment, which increases the overall cost associated with deploying the wireless communications network.
- timing packet(s) containing time information can be communicated among two or more nodes of the network.
- a particular node can receive a timing packet containing time information, and the particular node can update the time information in the received timing packet to reflect a delay associated with communication of the timing packet.
- the timing packet containing the updated time information can be sent to a second network node to allow for synchronization of the second network node.
- FIG. 1 is a block diagram of an example network of nodes and a time server that is able to transmit timing packets to one or more of the nodes, in accordance with an embodiment.
- FIG. 2 illustrates an example flow of a process to perform time synchronization and clock synchronization of a client node within a network, in accordance with an embodiment.
- FIGS. 3 and 4 illustrate example flows of determining an inter-node delay, in accordance with some embodiments.
- FIG. 5 illustrates an example flow for determining the delay through a node, in accordance with an embodiment.
- a mechanism is provided in a network of nodes to enable nodes to communicate timing packets containing time. information such that at least some of the nodes can be synchronized.
- Synchronizing a node refers to clock synchronizing the node and/or time synchronizing the node.
- Clock. synchronization refers to synchronizing a clock (as produced by a clock generator oscillator) in a node with respect to a reference or master clock.
- Time synchronization refers to synchronizing the time in the node with respect to a reference time.
- clock synchronization and/or time synchronization can be performed without the use of relatively expensive time reference devices, such as global positioning system (GPS) receivers, in each of the nodes of the network.
- the clock synchronization and/or time synchronization is based on communication of timing packets across nodes, where the timing packets contain time information that is updated to reflect various delays as the timing packets traverse through the nodes.
- Techniques according to some embodiments allow for relatively low cost (and low accuracy) clock devices to be used in the nodes while still being able to achieve accurate time and/or clock synchronization.
- a “clock device” refers to a device that produces a periodically oscillating signal (a “clock” or “clock signal”) having a frequency.
- each particular node is able to interact with its neighbor node(s) to determine inter-node delay(s) over network link(s) between the particular node and its neighbor node(s). Also, each node is able to measure the transport delay through the node.
- Such network nodes are referred to as “cognitive nodes” in that such nodes are aware of the network delay environment (including network link delays and node delays) in which the nodes are deployed. The nodes are able to learn about the network delay environment based on characterizations performed by the nodes. The characterizations can be performed periodically, or on an as-needed basis. As described further below, the characterizations can be accomplished by exchanges of timing packets among nodes.
- the inter-node delays being characterized are single-hop delays.
- a “single-hop delay” refers to the delay (over a network link) between two nodes with no intervening nodes in between. Since the inter-node delays being characterized are single-hop delays, such single-hop delays will not vary significantly with time and are generally not traffic dependent.
- a single-hop inter-node delay is generally relatively small (in other words, has a relatively short time duration). The relatively short time duration allows for the use of low stability clock devices in the nodes for performing relatively accurate synchronization, since the shortness of the inter-node delay means that there will be relatively small drift (in frequency) in the node clock devices during the time duration.
- the delay through the node is also relatively short such that node delay can also be accurately characterized using low stability clock 5 devices.
- the ability to perform synchronization of nodes using low stability (and low cost) clock devices in the nodes means that lower cost nodes can be provided, which can-reduce overall system cost.
- the frequency accuracy of the node clock device may be as poor as 100 ppm (parts per million); consequently, measurement of a 1 ms (millisecond) delay may introduce up to 100 ns (nanoseconds) time error that represents a worst case scenario, since typically inter-node delay time may be in the order of 0.1 ms resulting in just 10 ns time measurement error, or the frequency accuracy of the node clock device may be well under 100 ppm like 10 ppm resulting also in just 10 ns time measurement error. Provision of frequency synchronization to a primary reference clock (typically 10 parts-per-trillion frequency accuracy), such as in the case of Synchronous Ethernet removes the delay characterization error associated with the node clock frequency accuracy.
- a primary reference clock typically 10 parts-per-trillion frequency accuracy
- a delay table is maintained in each particular node, where the delay table specifies the inter-node delays between the particular node and its neighbor(s).
- the delay table can be periodically or intermittently updated based on changing conditions (e.g., such as changes in the surrounding temperature or other environment conditions).
- the nodes of the network can be considered clients of a time server that has a high-accuracy time reference source.
- the high-accuracy time reference source of the time server can be any of the following: a source based on global positioning system (GPS), a rubidium time clock source, a cesium time clock source, and so forth.
- GPS global positioning system
- the time reference source associated with the time server is of relatively high quality and accuracy.
- the network nodes, as clients of the time server are able to issue requests to the time server for timing packets. In response to such requests, the time server transmits timing packets to the requesting nodes, such that the nodes can perform synchronization.
- each of the nodes through which the timing packet traverses updates the time information contained in the timing packet to reflect network link delays (inter-node delays), as well as delays associated with traversal of a timing packet through each of the nodes in the path.
- inter-node delays network link delays
- delays associated with traversal of a timing packet through each of the nodes in the path At least some of the nodes can synchronize themselves in response to the timing packets.
- Such nodes are considered to be self-configuring.
- the nodes are able to automatically reconfigure in the event of network change, such as when network nodes are added or removed.
- time server Using a high-accuracy time reference source in the time server, rather than in each of the network nodes, allows for reduced system cost.
- time server it is noted that plural time servers can be used, such as plural time servers for plural corresponding groups of network nodes.
- FIG. 1 shows an example mobile or wireless communications system 100 that includes a number of nodes.
- Some of the nodes are base stations, such as base stations 102 , 104 , and 106 . Each base station 102 , 104 , and 106 is able to communicate over wireless links with mobile stations.
- the base stations 102 , 104 , and 106 are connected to respective base station controllers (or radio network controllers), not depicted in FIG. 1 , through a packet network including the switch nodes 108 - 120 .
- Other nodes in the mobile communications system 100 can be other types of nodes, such as packet data serving nodes (PDSNs), serving GPRS (General Packet Radio Service) support nodes, gateway GPRS support nodes, and so forth.
- PDSNs packet data serving nodes
- serving GPRS General Packet Radio Service
- the mobile communications system 100 also includes a time server 122 that is able to communicate with the other nodes of the system 100 .
- the time server 122 can be considered to be outside the system 100 . Also, alternatively, there can be multiple time servers in the system 100 .
- the time server 122 and network nodes 102 , 104 , 106 , 108 , 110 , 112 , 114 , 120 , 116 , 118 , and 120 can be interconnected by an asynchronous packet-switched network, such as an asynchronous Ethernet network, or other type of network. Since the underlying network is asynchronous, a synchronization mechanism or technique that provides timing packets can be used to allow for network nodes to be synchronized with each other. In other embodiments, a Synchronous Ethernet network, which provides physical layer frequency distribution, may be used.
- the time server 122 is associated with, or includes, a primary time reference source 124 and a timing packet generator 126 for generating timing packets.
- the primary time reference source 124 provides a relatively high accuracy reference time that can be inserted into timing packets to be sent by the time server 122 .
- the timing packet generator 126 can send timing packets in response to requests from clients (any of the network nodes in the system 100 ), or the timing packet generator 126 can send timing packets upon detection of one or more events.
- the timing packet generator 126 can be implemented in hardware, or alternatively, the timing packet generator 126 can be implemented with software that is executable on one or more central processing units (CPUs) 128 in the time server 122 .
- the CPU(s) is (are) connected to a storage device 130 in the time server 122 .
- the time server 122 also includes a primary or master clock device 127 that is synchronized with respect to the primary time reference source 124 .
- FIG. 1 illustrates an example timing packet 140 that can be transmitted by the time server 122 .
- the timing packet 140 illustrated in FIG. 1 is merely an example timing packet, as other timing packets can have other formats.
- the timing packet 140 includes a timestamp field 142 for storing an initial timestamp (corresponding to the time of the primary reference source 124 at the time point that the timing packet 140 is transmitted by the time server 122 ).
- the timing packet 140 also includes an end-to-end (E-E) timing packet (TP) indicator field 148 to indicate that the packet 140 is a timing packet (as opposed to other types of packets that can be communicated in the network 100 ).
- the TP indicator field 148 can also include a sequence number to allow for a particular client to determine a sequence of plural timing packets if plural timing packets are transmitted by the time server 122 to the particular client.
- the timing packet 140 also includes a network delay field 144 that is updated by nodes that the timing packet 140 has traversed through.
- a network delay field 144 that is updated by nodes that the timing packet 140 has traversed through.
- the timestamp field 142 in combination with network delay field 144 can be considered an example of “time information” in the timing packet that is updated.
- one field can be provided instead, with the one field updated to reflect cumulative delays as the timing packet traverses network links and nodes.
- the network delay field 144 is updated to reflect inter-node delays and delays through the nodes. Basically, as the timing packet 140 traverses through each particular node, the network delay value contained in the network delay field 144 is updated (by summing the existing network delay field 144 value with additional delay values, for example) to include the corresponding inter-node delay (from the upstream node to the particular node) and the delay through the particular node.
- the timing packet 140 also includes a last node ID field 146 that contains the identifier of the last node through which the timing packet 140 traversed. The last node ID field 146 is updated by each node that the timing packet passes through.
- nodes A, B, C, D, E, F, G are also labeled as nodes A, B, C, D, E, F, G.
- An example delay table 150 is depicted for node G (node 114 ). The delay table can be stored in a storage device 151 of the node 114 .
- the delay table (or other type of data structure) 150 includes six entries to represent the six inter-node delays between node G and its six neighbors.
- the inter-node delay between node A and node G (G-A) is ⁇ 1
- the inter-node delay between node B and node G (G-B) is ⁇ 6, and so forth.
- the delay through node G is represented as ⁇ SN .
- a respective delay table can be implemented in each of the nodes 102 , 104 , 106 , 108 , 110 , 112 , 114 , 116 , 118 , and 120 .
- the network node 114 also includes a clock device 160 (also referred to as a “client clock device” or “local clock device”) that produces the oscillating clock for the node 114 .
- the node 114 includes synchronization logic 162 (to perform synchronization based on timing packets) and characterization logic 164 (to characterize inter-node delays).
- the synchronization logic 162 is also able to update time information in timing packets to reflect network link delays and node delays as such timing packets pass through the node 114 .
- the synchronization logic 162 and characterization logic 164 can be implemented in hardware ⁇ e.g., microcontrollers, microprocessors, etc.) or in software executable on CPU(s). Updating of time information in a timing packet can be performed by the synchronization logic 162 implemented in hardware (layer 1) to enable for quicker response and more accurate time information updates.
- the clock device 160 , synchronization logic 162 , and characterization logic 164 can also be provided in the other nodes of FIG. 1 .
- FIG. 2 An example flow of synchronizing a client node 250 according to received timing packets is depicted in FIG. 2 .
- two timing packets 200 A and 202 A are depicted as being transmitted by the time server 122 through a data network 204 (e.g., Ethernet network).
- the timing packet 200 A is the first timing packet transmitted by the time server 122
- the timing packet 202 A is the second packet transmitted by the time server 122 after the first timing packet 200 A.
- additional one or more timing packets can be transmitted after timing packet 202 B.
- the first timing packet 200 A is also referred to as a start timing packet
- the second timing packet 202 A is referred to as a stop timing packet.
- the pair of timing packets 200 A, 202 A are used to perform clock synchronization of the clock device 160 in a client node 250 (which can be one of the base stations 102 , 104 , 106 of FIG. 1 or some other node, for example).
- Performing clock synchronization is accomplished by locking the frequency of the clock device 160 based on the start and stop timing packets 200 A, 202 A or can be done through physical-layer techniques like Synchronous Ethernet.
- the timestamp field 142 of each of timing packets 200 A, 202 A contains the corresponding timestamp representing the time at which the respective timing packet was transmitted by the time server 122 .
- the timestamp field 142 for the start timing packet 200 A is 1:00 pm
- the timestamp field 142 for the stop timing packet 202 A is 1:01 pm (sent one minute later).
- the network delay field 144 for each of timing packets 200 A, 202 A has the value zero since the timing packets have just been transmitted by the time server 122 and thus have not experienced network delays.
- the last node ID field 146 for each of timing packets 200 A, 202 A indicates “node T,” which represents the time server 122 .
- the timing packet indicator field 148 of each of the timing packets 200 A, 202 A contains a respective sequence number to indicate the sequence of the timing packets 200 A, 202 A. For example, the timing packet indicator field 148 of the timing packet 200 A has sequence number 1, whereas the timing packet indicator field 148 of the stop timing packet 202 A has sequence number 2.
- timing 20 packets After traversing through various nodes of the data network 204 , the timing 20 packets become timing packets 200 B, 202 B, whose contents have been updated.
- the timestamp fields 142 of the timing packets 200 B, 202 B remain unchanged, as they represent the original transmit time of the time server 122 .
- the network delay fields 144 of the timing packets 200 B, 202 B have been updated to reflect the cumulative delays experienced by respective timing packets 200 B, 202 B as such timing packets traversed through nodes of the data network 204 .
- the network delay field 144 of the start timing packet 200 B has been updated to reflect a network delay of 0.001 seconds
- the network delay field 144 of the stop timing packet 202 B has been updated to reflect a network delay of 0.002 seconds.
- the timing packets 200 B, 202 B have experienced different delays through the data network 204 , which can result from the timing packets traversing through different network paths, and/or experiencing different queuing delays in the nodes of the data network 204 .
- the last node ID fields 146 of the timing packets 200 B, 202 B have also been updated to reflect the identifier of the last node, which in this example is “node N.”
- the timing packets 200 B, 202 B are received by the client node 250 .
- the inter-arrival time delay can be calculated (at 208 ) by the client node 250 from the difference in the timestamp values and the network delay values of the timestamp fields 142 and network delay fields 144 , respectively.
- the inter-arrival delay of the timing packets 200 B, 202 B is calculated by first summing the timestamp value and the network delay value of the start timing packet 200 B to produce a first time value, and by summing the timestamp value and network delay value of the stop timing packet 202 B to produce a second time value.
- the calculated inter-arrival delay represents the actual delay based on when the timing packets 200 B, 202 B were originally transmitted by the time server 122 and the delays experienced by the timing packets 200 B, 202 B as they traversed through the data network 204 .
- This calculated inter-arrival delay is to be compared to a measured inter-arrival time, as measured (at 210 ) by the client node 250 .
- the measured delay is based on identifying the start time upon receipt of the start timing packet 200 B by the client node 250 , and identifying the stop time upon receipt of the stop timing packet 202 B by the client node 250 .
- the difference between the start and stop times is the measured inter-arrival time.
- Measuring the inter-arrival time can be accomplished by using a counter that starts upon receipt of the start timing packet 200 B, and stops upon receipt of the stop timing packet 202 B. The count value in the counter would then indicate the inter-arrival time.
- the calculated inter-arrival time ( 208 ) is 60.001 seconds (which represents the 1 minute difference in original transmission of the timing packets 200 A, 202 A by the time server 122 , and the 0.001 second difference in network delays experienced by the timing packets 200 B, 202 B).
- the measured inter-arrival time ( 210 ) is 60.0011 seconds.
- the difference (time error) between the CA measured inter-arrival time and the calculated inter-arrival time is calculated (at 212 ), which in the example of FIG. 2 is 0.0001 second. Effectively, the difference between the measured inter-arrival time and the calculated inter-arrival time is due to an error of the client node clock device 160 to the primary clock device 127 ( FIG. 1 ) in the time server 122 and also network traffic characteristics although the impact of the latter is mitigated by averaging in control block 214 .
- a procedure to adjust the client clock device 160 using the time error value is then initiated (at 214 ).
- Intrinsic to the client clock adjustment procedure is averaging of the error signal to remove random packet delay variation.
- a frequency lock detect is performed (at 216 ) based on comparing the time error value to an error signal threshold. If the time error value is greater than or equal to the threshold, then frequency lock of the client clock device 160 is to be performed. On the other hand, if the time error value is less than the error signal threshold, then frequency lock does not have to be performed.
- a client node clock control loop is performed (at 218 ) to perform adjustment of the frequency of the clock device 160 according to the time error value. This effectively adjusts the frequency of the clock device 160 to be the same as the primary clock device 127 of the time server 122 that is based on the reference source 124 . In other words, the clock device 160 frequency is locked to the primary clock device 127 frequency. It should be noted that frequency lock can be achieved by other methods such as Synchronous Ethernet.
- time synchronization Another type of synchronization that can be performed is time synchronization, which is performed after the clock device 160 has been adjusted (locked to the frequency of time server primary clock device).
- the frequency lock of the clock device 160 at the client node 250 is indicated by setting a frequency lock flag in the client node clock control loop ( 218 ). If the frequency lock flag is set, then the timestamp of the next timing packet (received after timing packet 202 B) is used to synchronize (at 220 ) the time of the client node 250 . Basically, the timestamp and network delay of the next timing packet are used to set the initial time (time epoch) of the client node 250 .
- the sum of the timestamp and the network delay of this next timing packet provides a time value that is used as the initial time (epoch).
- the time of day of the client node 250 is then subsequently incremented (at 222 ) using the client clock device 160 .
- the tasks 208 , 210 , 212 , 214 , 216 , and 220 in FIG. 2 can be performed by the synchronization logic ( 162 in FIG. 1 ) of the client node 250 .
- FIG. 3 illustrates an example flow of calculating an inter-node delay (between node A and node B). In other implementations, other techniques of calculating inter-node delays can be used.
- the inter-node delay calculation is controlled by characterization logic 164 in each of the nodes A and B ( 164 A in node A, and 164 B in node B).
- the characterization logic 164 A cooperates with a packet server 300 A and a timing packet delay counter 302 A in node A.
- the packet server 300 A of node A is used to transmit a characterization timing packet 303 A to node B.
- the characterization logic 164 A causes the timing packet delay counter 302 A in node A to start.
- the timing packet 303 A is sent across the network link between nodes A and B, which is received by the characterization logic 164 B of node B.
- the characterization logic 164 B of node B Upon receipt of the timing packet 303 A, the characterization logic 164 B of node B starts a node delay counter 306 B in node B.
- the node delay counter 306 B is used to calculate the delay in propagating the characterization timing packet through node B.
- the delay counted by the node delay counter 306 B is used to update the characterization timing packet received by node B, with the updated characterization timing packet referred to as 303 B.
- the packet server 300 B of node B sends the updated characterization timing packet ( 303 B) back to node A.
- the updated characterization timing packet 303 B contains a network delay field that contains the value of the delay through node B.
- the updated characterization timing packet 303 B also contains a node ID field (which would contain the identifier of node B), as well as a node-to-node (N-N) TP Indicator field to indicate that the packet 303 B is a 10 timing packet.
- N-N node-to-node
- the characterization timing packet 303 B is shown to have fields that differ slightly from the timing packet depicted in FIG. 2 , it is noted that in some implementations, the characterization timing packet (as generated by a node during characterization) has the same format as a timing packet sent by the time server 122 ( FIG. 2 ). However, in other implementations, the formats of the timing packet issued by the time server 122 and the packet servers of nodes can be different.
- the characterization logic 164 A in node A causes the timing packet delay counter 302 A to stop.
- the count of the timing packet delay counter 302 A is converted into a time value that represents the roundtrip delay between nodes A and B, where the roundtrip delay includes the delay through node B.
- the roundtrip network link delay is calculated (at 310 ) by subtracting the node B delay (contained in the node B delay field of timing packet 303 B) from the roundtrip delay.
- the inter-node delay is then calculated (at 312 ) as being half the roundtrip network link delay.
- the node A delay table is then updated (at 314 ) using the inter-node delay calculated at 312 .
- FIG. 4 shows an alternative technique of calculating the inter-node delay between nodes A and B.
- the technique of FIG. 4 is a packet mirror technique. Hardware in node B is able to “bounce” a characterization timing packet back to node A without having to buffer or queue the characterization timing packet. In such a technique, the processing performed at node B subjects the characterization timing packet to negligible node delay in node B. Thus, use of the node delay counter 306 B as performed in FIG. 3 can be avoided.
- the characterization logic 164 A causes the packet server 300 A of node A to transmit a characterization timing packet 402 A.
- the timing packet delay counter 302 A in node A is started.
- the timing packet 402 A is transmitted across the network link between nodes A and B.
- the characterization timing packet 402 is received by a physical layer 404 in node B. Above the physical layer 404 is a media access control (MAC) layer 406 . Also in node B is a controller 408 , which can be implemented with a field programmable gate array (FPGA) device or other device, that is able to recognize that the characterization timing packet 402 is a timing packet. Upon receipt of the characterization timing packet 402 , controller 408 is able to reflect (or loop) the characterization timing packet 402 back to node A over the network link without queuing the timing packet 402 . The amount of time that the characterization timing packet spends in node B is the time to read the TP indicator field of the timing packet (while the controller 408 processes the characterization timing packet).
- MAC media access control
- the characterization logic 164 A in node A Upon receipt of the characterization timing packet 402 from node B, the characterization logic 164 A in node A causes the timing packet delay counter 302 A to stop counting.
- the timing packet delay counter value corresponds to the roundtrip network link delay, which is set (at 410 ).
- the inter-node delay is then calculated (at 412 ) by dividing the roundtrip network link delay in half.
- the delay table of node A is then 25 updated (at 414 ) with the inter-node delay.
- FIG. 5 illustrates an example of how a node determines the delay associated with a timing packet traversing through the node.
- the node delay is computed as a timing packet issued by the time server 122 ( FIG. 2 ) makes its way through the node to a target client node.
- a timing packet 140 A is received by a node.
- the synchronization logic 162 of the node starts the node delay counter 306 .
- the node delay counter 306 is stopped when the timing packet is ready to be transmitted from the node.
- the synchronization logic 162 updates the network delay field of the timing packet 140 A, and causes the packet server 310 to transmit the updated timing packet MOB to the next node.
- the various tasks performed by the synchronization logic 162 and characterization logic 164 are performed at the hardware level for enhanced processing speeds.
- the synchronization logic 162 and characterization logic 164 can be implemented in software on a processor.
- the processor includes microprocessors, microcontrollers, processor modules or subsystems (including one or more microprocessors or microcontrollers), or other control or computing devices.
- a “processor” can refer to a single component or to plural components.
- Data and instructions (of the software) are stored in respective storage devices, which are implemented as one or more computer-readable or computer-usable storage media.
- the storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact disks (CDs) or digital video disks (DVDs).
- DRAMs or SRAMs dynamic or static random access memories
- EPROMs erasable and programmable read-only memories
- EEPROMs electrically erasable and programmable read-only memories
- flash memories magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape
- optical media such as compact disks (CDs) or digital video disks (DVDs).
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Synchronisation In Digital Transmission Systems (AREA)
Abstract
A first node in a network receives a timing packet containing time information, where the timing packet is originated by a time server. The first node then updates the time information in the timing packet to reflect a delay associated with communicating the timing packet over a network link. The first node then updates the time information in the timing packet to reflect the delay associated transfer of the timing packet through the node. The first node sends the timing packet with the updated time information to a second node to enable the second node to use the updated time information for synchronization of the second node. The process repeats across an arbitrary number of nodes to enable time alignment between the first node and final destination node.
Description
- This application is a division of U.S. application Ser. No. 12/006,151, filed Dec. 31, 2007, the entire contents of which is incorporated fully by reference herein.
- The invention relates generally to communicating time information across nodes in a network to enable synchronization of one or more of the nodes.
- In certain applications, nodes within a network have to be time-synchronized. One example of such an application involves a wireless or mobile communications network, such as a CDMA (code division multiple access) 2000 and WiMAX (World Interoperability for Microwave Access), that has nodes such as base stations that are time synchronized to enable high-speed communications with mobile stations. Typically, due to strict requirements of accurate time references by the wireless standards, nodes within a mobile communications network have used global position system (GPS) receivers to maintain time synchronization. However, GPS receivers or other high accuracy time reference sources provided in each node add to the hardware and installation cost of network equipment, which increases the overall cost associated with deploying the wireless communications network.
- In general, according to an embodiment, to enable synchronization of at least one node in a network, timing packet(s) containing time information can be communicated among two or more nodes of the network. A particular node can receive a timing packet containing time information, and the particular node can update the time information in the received timing packet to reflect a delay associated with communication of the timing packet. The timing packet containing the updated time information can be sent to a second network node to allow for synchronization of the second network node.
- Other or alternative features will become apparent from the following description, from the drawings, and from the claims.
-
FIG. 1 is a block diagram of an example network of nodes and a time server that is able to transmit timing packets to one or more of the nodes, in accordance with an embodiment. -
FIG. 2 illustrates an example flow of a process to perform time synchronization and clock synchronization of a client node within a network, in accordance with an embodiment. -
FIGS. 3 and 4 illustrate example flows of determining an inter-node delay, in accordance with some embodiments. -
FIG. 5 illustrates an example flow for determining the delay through a node, in accordance with an embodiment. - In the following description, numerous details are set forth to provide an understanding of some embodiments. However, it will be understood by those skilled in the art that some embodiments may be practiced without these details and that numerous variations or modifications from the described embodiments may be possible.
- In accordance with some embodiments, a mechanism is provided in a network of nodes to enable nodes to communicate timing packets containing time. information such that at least some of the nodes can be synchronized. Synchronizing a node refers to clock synchronizing the node and/or time synchronizing the node. Clock. synchronization refers to synchronizing a clock (as produced by a clock generator oscillator) in a node with respect to a reference or master clock. Time synchronization refers to synchronizing the time in the node with respect to a reference time.
- In accordance with some embodiments, clock synchronization and/or time synchronization can be performed without the use of relatively expensive time reference devices, such as global positioning system (GPS) receivers, in each of the nodes of the network. The clock synchronization and/or time synchronization is based on communication of timing packets across nodes, where the timing packets contain time information that is updated to reflect various delays as the timing packets traverse through the nodes. Techniques according to some embodiments allow for relatively low cost (and low accuracy) clock devices to be used in the nodes while still being able to achieve accurate time and/or clock synchronization. As used here, a “clock device” refers to a device that produces a periodically oscillating signal (a “clock” or “clock signal”) having a frequency.
- In some embodiments, each particular node is able to interact with its neighbor node(s) to determine inter-node delay(s) over network link(s) between the particular node and its neighbor node(s). Also, each node is able to measure the transport delay through the node. Such network nodes are referred to as “cognitive nodes” in that such nodes are aware of the network delay environment (including network link delays and node delays) in which the nodes are deployed. The nodes are able to learn about the network delay environment based on characterizations performed by the nodes. The characterizations can be performed periodically, or on an as-needed basis. As described further below, the characterizations can be accomplished by exchanges of timing packets among nodes.
- In some embodiments, the inter-node delays being characterized are single-hop delays. A “single-hop delay” refers to the delay (over a network link) between two nodes with no intervening nodes in between. Since the inter-node delays being characterized are single-hop delays, such single-hop delays will not vary significantly with time and are generally not traffic dependent. A single-hop inter-node delay is generally relatively small (in other words, has a relatively short time duration). The relatively short time duration allows for the use of low stability clock devices in the nodes for performing relatively accurate synchronization, since the shortness of the inter-node delay means that there will be relatively small drift (in frequency) in the node clock devices during the time duration. The delay through the node is also relatively short such that node delay can also be accurately characterized using low stability clock 5 devices. The ability to perform synchronization of nodes using low stability (and low cost) clock devices in the nodes means that lower cost nodes can be provided, which can-reduce overall system cost.
- For example, the frequency accuracy of the node clock device may be as poor as 100 ppm (parts per million); consequently, measurement of a 1 ms (millisecond) delay may introduce up to 100 ns (nanoseconds) time error that represents a worst case scenario, since typically inter-node delay time may be in the order of 0.1 ms resulting in just 10 ns time measurement error, or the frequency accuracy of the node clock device may be well under 100 ppm like 10 ppm resulting also in just 10 ns time measurement error. Provision of frequency synchronization to a primary reference clock (typically 10 parts-per-trillion frequency accuracy), such as in the case of Synchronous Ethernet removes the delay characterization error associated with the node clock frequency accuracy.
- In some embodiments, a delay table is maintained in each particular node, where the delay table specifies the inter-node delays between the particular node and its neighbor(s). The delay table can be periodically or intermittently updated based on changing conditions (e.g., such as changes in the surrounding temperature or other environment conditions).
- The nodes of the network can be considered clients of a time server that has a high-accuracy time reference source. The high-accuracy time reference source of the time server can be any of the following: a source based on global positioning system (GPS), a rubidium time clock source, a cesium time clock source, and so forth. In general, the time reference source associated with the time server is of relatively high quality and accuracy. The network nodes, as clients of the time server, are able to issue requests to the time server for timing packets. In response to such requests, the time server transmits timing packets to the requesting nodes, such that the nodes can perform synchronization. As the timing packet traverses nodes along a path through the network, each of the nodes through which the timing packet traverses updates the time information contained in the timing packet to reflect network link delays (inter-node delays), as well as delays associated with traversal of a timing packet through each of the nodes in the path. In this manner, at least some of the nodes can synchronize themselves in response to the timing packets. Such nodes are considered to be self-configuring. Moreover, the nodes are able to automatically reconfigure in the event of network change, such as when network nodes are added or removed.
- Using a high-accuracy time reference source in the time server, rather than in each of the network nodes, allows for reduced system cost. Although just one time server is mentioned, it is noted that plural time servers can be used, such as plural time servers for plural corresponding groups of network nodes.
-
FIG. 1 shows an example mobile orwireless communications system 100 that includes a number of nodes. Some of the nodes are base stations, such asbase stations base station base stations FIG. 1 , through a packet network including the switch nodes 108-120. Other nodes in themobile communications system 100 can be other types of nodes, such as packet data serving nodes (PDSNs), serving GPRS (General Packet Radio Service) support nodes, gateway GPRS support nodes, and so forth. - As further depicted in
FIG. 1 , themobile communications system 100 also includes atime server 122 that is able to communicate with the other nodes of thesystem 100. In different implementations, thetime server 122 can be considered to be outside thesystem 100. Also, alternatively, there can be multiple time servers in thesystem 100. - The
time server 122 andnetwork nodes - Although reference has been made to a mobile communications system, it is contemplated that techniques according to some embodiments can also be applied to other types of systems.
- The
time server 122 is associated with, or includes, a primarytime reference source 124 and atiming packet generator 126 for generating timing packets. The primarytime reference source 124 provides a relatively high accuracy reference time that can be inserted into timing packets to be sent by thetime server 122. Thetiming packet generator 126 can send timing packets in response to requests from clients (any of the network nodes in the system 100), or thetiming packet generator 126 can send timing packets upon detection of one or more events. Thetiming packet generator 126 can be implemented in hardware, or alternatively, thetiming packet generator 126 can be implemented with software that is executable on one or more central processing units (CPUs) 128 in thetime server 122. The CPU(s) is (are) connected to astorage device 130 in thetime server 122. Thetime server 122 also includes a primary ormaster clock device 127 that is synchronized with respect to the primarytime reference source 124. -
FIG. 1 illustrates anexample timing packet 140 that can be transmitted by thetime server 122. Note that thetiming packet 140 illustrated inFIG. 1 is merely an example timing packet, as other timing packets can have other formats. Thetiming packet 140 includes atimestamp field 142 for storing an initial timestamp (corresponding to the time of theprimary reference source 124 at the time point that thetiming packet 140 is transmitted by the time server 122). Thetiming packet 140 also includes an end-to-end (E-E) timing packet (TP)indicator field 148 to indicate that thepacket 140 is a timing packet (as opposed to other types of packets that can be communicated in the network 100). TheTP indicator field 148 can also include a sequence number to allow for a particular client to determine a sequence of plural timing packets if plural timing packets are transmitted by thetime server 122 to the particular client. - The
timing packet 140 also includes anetwork delay field 144 that is updated by nodes that thetiming packet 140 has traversed through. Note that thetimestamp field 142 in combination withnetwork delay field 144 can be considered an example of “time information” in the timing packet that is updated. Alternatively, instead of providing aseparate timestamp field 142 andnetwork delay field 144, one field can be provided instead, with the one field updated to reflect cumulative delays as the timing packet traverses network links and nodes. - The
network delay field 144 is updated to reflect inter-node delays and delays through the nodes. Basically, as thetiming packet 140 traverses through each particular node, the network delay value contained in thenetwork delay field 144 is updated (by summing the existingnetwork delay field 144 value with additional delay values, for example) to include the corresponding inter-node delay (from the upstream node to the particular node) and the delay through the particular node. Thetiming packet 140 also includes a lastnode ID field 146 that contains the identifier of the last node through which thetiming packet 140 traversed. The lastnode ID field 146 is updated by each node that the timing packet passes through. - As depicted in the example
FIG. 1 , some of the various nodes are also labeled as nodes A, B, C, D, E, F, G. An example delay table 150 is depicted for node G (node 114). The delay table can be stored in astorage device 151 of thenode 114. - The delay table (or other type of data structure) 150 includes six entries to represent the six inter-node delays between node G and its six neighbors. The inter-node delay between node A and node G (G-A) is τ1, the inter-node delay between node B and node G (G-B) is τ6, and so forth. The delay through node G is represented as τSN. Note that a respective delay table can be implemented in each of the
nodes - In the example of
FIG. 1 , thenetwork node 114 also includes a clock device 160 (also referred to as a “client clock device” or “local clock device”) that produces the oscillating clock for thenode 114. Also, thenode 114 includes synchronization logic 162 (to perform synchronization based on timing packets) and characterization logic 164 (to characterize inter-node delays). Thesynchronization logic 162 is also able to update time information in timing packets to reflect network link delays and node delays as such timing packets pass through thenode 114. Thesynchronization logic 162 andcharacterization logic 164 can be implemented in hardware {e.g., microcontrollers, microprocessors, etc.) or in software executable on CPU(s). Updating of time information in a timing packet can be performed by thesynchronization logic 162 implemented in hardware (layer 1) to enable for quicker response and more accurate time information updates. - The
clock device 160,synchronization logic 162, andcharacterization logic 164 can also be provided in the other nodes ofFIG. 1 . - An example flow of synchronizing a
client node 250 according to received timing packets is depicted inFIG. 2 . In the example ofFIG. 2 , two timingpackets time server 122 through a data network 204 (e.g., Ethernet network). Thetiming packet 200A is the first timing packet transmitted by thetime server 122, and thetiming packet 202A is the second packet transmitted by thetime server 122 after thefirst timing packet 200A. Although not shown, additional one or more timing packets can be transmitted after timingpacket 202B. Thefirst timing packet 200A is also referred to as a start timing packet, and thesecond timing packet 202A is referred to as a stop timing packet. The pair of timingpackets clock device 160 in a client node 250 (which can be one of thebase stations FIG. 1 or some other node, for example). Performing clock synchronization is accomplished by locking the frequency of theclock device 160 based on the start and stop timingpackets - As transmitted by the
time server 122, thetimestamp field 142 of each of timingpackets time server 122. Thus, for example, thetimestamp field 142 for thestart timing packet 200A is 1:00 pm, and thetimestamp field 142 for thestop timing packet 202A is 1:01 pm (sent one minute later). - The
network delay field 144 for each of timingpackets time server 122 and thus have not experienced network delays. The lastnode ID field 146 for each of timingpackets time server 122. The timingpacket indicator field 148 of each of thetiming packets timing packets packet indicator field 148 of thetiming packet 200A hassequence number 1, whereas the timingpacket indicator field 148 of thestop timing packet 202A hassequence number 2. - After traversing through various nodes of the
data network 204, the timing 20 packets become timingpackets packets time server 122. However, the network delay fields 144 of the timingpackets respective timing packets data network 204. In the example ofFIG. 2 , after traversal through thedata network 204, thenetwork delay field 144 of thestart timing packet 200B has been updated to reflect a network delay of 0.001 seconds, whereas thenetwork delay field 144 of thestop timing packet 202B has been updated to reflect a network delay of 0.002 seconds. Note that the timingpackets data network 204, which can result from the timing packets traversing through different network paths, and/or experiencing different queuing delays in the nodes of thedata network 204. The last node ID fields 146 of the timingpackets - The timing
packets client node 250. From the timingpackets client node 250 from the difference in the timestamp values and the network delay values of the timestamp fields 142 and network delay fields 144, respectively. Effectively, the inter-arrival delay of the timingpackets start timing packet 200B to produce a first time value, and by summing the timestamp value and network delay value of thestop timing packet 202B to produce a second time value. Then, the difference between the second time value and the first time value is calculated to produce the inter-arrival delay. The calculated inter-arrival delay represents the actual delay based on when the timingpackets time server 122 and the delays experienced by the timingpackets data network 204. - This calculated inter-arrival delay is to be compared to a measured inter-arrival time, as measured (at 210) by the
client node 250. The measured delay is based on identifying the start time upon receipt of thestart timing packet 200B by theclient node 250, and identifying the stop time upon receipt of thestop timing packet 202B by theclient node 250. The difference between the start and stop times is the measured inter-arrival time. Measuring the inter-arrival time can be accomplished by using a counter that starts upon receipt of thestart timing packet 200B, and stops upon receipt of thestop timing packet 202B. The count value in the counter would then indicate the inter-arrival time. - In the example of
FIG. 2 , the calculated inter-arrival time (208) is 60.001 seconds (which represents the 1 minute difference in original transmission of thetiming packets time server 122, and the 0.001 second difference in network delays experienced by the timingpackets FIG. 2 is 0.0001 second. Effectively, the difference between the measured inter-arrival time and the calculated inter-arrival time is due to an error of the clientnode clock device 160 to the primary clock device 127 (FIG. 1 ) in thetime server 122 and also network traffic characteristics although the impact of the latter is mitigated by averaging incontrol block 214. - A procedure to adjust the
client clock device 160 using the time error value is then initiated (at 214). Intrinsic to the client clock adjustment procedure is averaging of the error signal to remove random packet delay variation. - Next, a frequency lock detect is performed (at 216) based on comparing the time error value to an error signal threshold. If the time error value is greater than or equal to the threshold, then frequency lock of the
client clock device 160 is to be performed. On the other hand, if the time error value is less than the error signal threshold, then frequency lock does not have to be performed. In response to detecting that the time error value is greater than or equal to the error signal threshold, a client node clock control loop is performed (at 218) to perform adjustment of the frequency of theclock device 160 according to the time error value. This effectively adjusts the frequency of theclock device 160 to be the same as theprimary clock device 127 of thetime server 122 that is based on thereference source 124. In other words, theclock device 160 frequency is locked to theprimary clock device 127 frequency. It should be noted that frequency lock can be achieved by other methods such as Synchronous Ethernet. - The above has described the process for performing clock synchronization of the
clock device 160 of theclient node 250. Another type of synchronization that can be performed is time synchronization, which is performed after theclock device 160 has been adjusted (locked to the frequency of time server primary clock device). The frequency lock of theclock device 160 at theclient node 250 is indicated by setting a frequency lock flag in the client node clock control loop (218). If the frequency lock flag is set, then the timestamp of the next timing packet (received after timingpacket 202B) is used to synchronize (at 220) the time of theclient node 250. Basically, the timestamp and network delay of the next timing packet are used to set the initial time (time epoch) of theclient node 250. The sum of the timestamp and the network delay of this next timing packet provides a time value that is used as the initial time (epoch). The time of day of theclient node 250 is then subsequently incremented (at 222) using theclient clock device 160. - The
tasks FIG. 2 can be performed by the synchronization logic (162 inFIG. 1 ) of theclient node 250. -
FIG. 3 illustrates an example flow of calculating an inter-node delay (between node A and node B). In other implementations, other techniques of calculating inter-node delays can be used. - The inter-node delay calculation is controlled by
characterization logic 164 in each of the nodes A and B (164 A in node A, and 164B in node B). Thecharacterization logic 164A cooperates with apacket server 300A and a timingpacket delay counter 302A in node A. Under control of thecharacterization logic 164A, thepacket server 300A of node A is used to transmit acharacterization timing packet 303A to node B. When thetiming packet 303A is transmitted by thepacket server 300A, thecharacterization logic 164A causes the timingpacket delay counter 302A in node A to start. - The
timing packet 303A is sent across the network link between nodes A and B, which is received by thecharacterization logic 164B of node B. Upon receipt of thetiming packet 303A, thecharacterization logic 164B of node B starts anode delay counter 306B in node B. Thenode delay counter 306B is used to calculate the delay in propagating the characterization timing packet through node B. The delay counted by thenode delay counter 306B is used to update the characterization timing packet received by node B, with the updated characterization timing packet referred to as 303B. Thepacket server 300B of node B sends the updated characterization timing packet (303B) back to node A. The updatedcharacterization timing packet 303B contains a network delay field that contains the value of the delay through node B. The updatedcharacterization timing packet 303B also contains a node ID field (which would contain the identifier of node B), as well as a node-to-node (N-N) TP Indicator field to indicate that thepacket 303B is a 10 timing packet. Although thecharacterization timing packet 303B is shown to have fields that differ slightly from the timing packet depicted inFIG. 2 , it is noted that in some implementations, the characterization timing packet (as generated by a node during characterization) has the same format as a timing packet sent by the time server 122 (FIG. 2 ). However, in other implementations, the formats of the timing packet issued by thetime server 122 and the packet servers of nodes can be different. - Upon receipt of the updated
characterization timing packet 303B, thecharacterization logic 164A in node A causes the timingpacket delay counter 302A to stop. The count of the timingpacket delay counter 302A is converted into a time value that represents the roundtrip delay between nodes A and B, where the roundtrip delay includes the delay through node B. To obtain the true roundtrip network link delay (which does not include the delay through node B), the roundtrip network link delay is calculated (at 310) by subtracting the node B delay (contained in the node B delay field of timingpacket 303B) from the roundtrip delay. The inter-node delay is then calculated (at 312) as being half the roundtrip network link delay. The node A delay table is then updated (at 314) using the inter-node delay calculated at 312. -
FIG. 4 shows an alternative technique of calculating the inter-node delay between nodes A and B. The technique ofFIG. 4 is a packet mirror technique. Hardware in node B is able to “bounce” a characterization timing packet back to node A without having to buffer or queue the characterization timing packet. In such a technique, the processing performed at node B subjects the characterization timing packet to negligible node delay in node B. Thus, use of thenode delay counter 306B as performed inFIG. 3 can be avoided. - As depicted in
FIG. 4 , thecharacterization logic 164A causes thepacket server 300A of node A to transmit a characterization timing packet 402A. Upon transmission of the timing packet 402A, the timingpacket delay counter 302A in node A is started. The timing packet 402A is transmitted across the network link between nodes A and B. - The
characterization timing packet 402 is received by aphysical layer 404 in node B. Above thephysical layer 404 is a media access control (MAC)layer 406. Also in node B is acontroller 408, which can be implemented with a field programmable gate array (FPGA) device or other device, that is able to recognize that thecharacterization timing packet 402 is a timing packet. Upon receipt of thecharacterization timing packet 402,controller 408 is able to reflect (or loop) thecharacterization timing packet 402 back to node A over the network link without queuing thetiming packet 402. The amount of time that the characterization timing packet spends in node B is the time to read the TP indicator field of the timing packet (while thecontroller 408 processes the characterization timing packet). - Upon receipt of the
characterization timing packet 402 from node B, thecharacterization logic 164A in node A causes the timingpacket delay counter 302A to stop counting. The timing packet delay counter value corresponds to the roundtrip network link delay, which is set (at 410). The inter-node delay is then calculated (at 412) by dividing the roundtrip network link delay in half. The delay table of node A is then 25 updated (at 414) with the inter-node delay. -
FIG. 5 illustrates an example of how a node determines the delay associated with a timing packet traversing through the node. Note that the node delay is computed as a timing packet issued by the time server 122 (FIG. 2 ) makes its way through the node to a target client node. As depicted inFIG. 5 , atiming packet 140A is received by a node. In response to receipt of thetiming packet 140A, thesynchronization logic 162 of the node starts thenode delay counter 306. Thenode delay counter 306 is stopped when the timing packet is ready to be transmitted from the node. At that point, thesynchronization logic 162 updates the network delay field of thetiming packet 140 A, and causes thepacket server 310 to transmit the updated timing packet MOB to the next node. - As noted above, the various tasks performed by the
synchronization logic 162 andcharacterization logic 164 are performed at the hardware level for enhanced processing speeds. However, in other implementations, thesynchronization logic 162 andcharacterization logic 164 can be implemented in software on a processor. - Instructions of such software are executed on the processor. The processor includes microprocessors, microcontrollers, processor modules or subsystems (including one or more microprocessors or microcontrollers), or other control or computing devices. A “processor” can refer to a single component or to plural components.
- Data and instructions (of the software) are stored in respective storage devices, which are implemented as one or more computer-readable or computer-usable storage media. The storage media include different forms of memory including semiconductor memory devices such as dynamic or static random access memories (DRAMs or SRAMs), erasable and programmable read-only memories (EPROMs), electrically erasable and programmable read-only memories (EEPROMs) and flash memories; magnetic disks such as fixed, floppy and removable disks; other magnetic media including tape; and optical media such as compact disks (CDs) or digital video disks (DVDs).
- In the foregoing description, numerous details are set forth to provide an understanding of the present invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these details. While the invention has been disclosed with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover such modifications and variations as fall within the true spirit and scope of the invention.
- Although the present invention has been described with respect to a specific preferred embodiment thereof, various changes and modifications may be suggested to one skilled in the art and it is intended that the present invention encompass such changes and modifications as fall within the scope of the appended claims.
Claims (20)
1. A method comprising:
receiving, at a first node in a network, a timing packet comprising time information, the timing packet being originated by a time server;
updating, by the first node, the time information in the timing packet to indicate a cumulative delay comprising:
a delay associated with communicating the timing packet to the first node over a network link to which the first node is connected; and
a delay experienced by the timing packet in transit through the first node determined using a local clock of the first node, the local clock of the first node having a lower accuracy than a clock associated with the time server; and
sending, by the first node, the timing packet with the updated time information to a second node to enable the second node to use the updated time information for synchronization of the second node.
2. The method of claim 1 , wherein the time information in the received timing packet comprises information indicating a time at which the timing packet was originated by the time server.
3. The method of claim 2 , wherein:
the timing packet is received from a third node of the network; and
the time information in the received timing packet comprises information indicating a cumulative delay experienced by the timing packet before transmission of the timing packet from the third node to the first node.
4. The method of claim 3 , wherein the delay associated with communicating the timing packet to the first node over a network link to which the first node is connected is a delay associated with communicating the timing packet over a network link connecting the third node to the first node.
5. The method of claim 1 , wherein the timing packet comprises information indicating that the timing packet is a timing packet.
6. The method of claim 1 , comprising:
receiving the timing packet with the updated timing information at the second node; and
using the updated time information to synchronize the second node.
7. The method of claim 6 , wherein using the updated time information to synchronize the second node comprises using the updated time information to synchronize a local clock of the second node to a reference time source of the time server.
8. The method of claim 6 , wherein using the updated time information to synchronize the second node comprises determining a time epoch for the second node.
9. The method of claim 1 , wherein the first node maintains a delay table, the delay table indicating respective delays associated with network links to which the first node is connected.
10. The method of claim 1 , wherein the first node is a packet network node and the second node is a base station of a mobile communications system connected to the packet network node.
11. A system comprising, at a first node of a communications network:
a local clock; and
characterization logic adapted:
to receive a timing packet comprising time information, the timing packet being originated by a time server;
to update the time information in the timing packet to indicate a cumulative delay comprising:
a delay associated with communicating the timing packet to the first node over a network link to which the first node is connected; and
a delay experienced by the timing packet in transit through the first node determined using the local clock of the first node, the local clock of the first node having a lower accuracy than a clock associated with the time server; and
to send the timing packet with the updated time information to a second node to enable the second node to use the updated time information for synchronization of the second node.
12. The system of claim 11 , wherein the time information in the received timing packet comprises information indicating a time at which the timing packet was originated by the time server.
13. The system of claim 12 , wherein:
the timing packet is received from a third node of the network; and
the time information in the received timing packet comprises information indicating a cumulative delay experienced by the timing packet before transmission of the timing packet from the third node to the first node.
14. The system of claim 13 , wherein the delay associated with communicating the timing packet to the first node over a network link to which the first node is connected is a delay associated with communicating the timing packet over a network link connecting the third node to the first node.
15. The system of claim 11 , wherein the timing packet comprises information indicating that the timing packet is a timing packet.
16. The system of claim 11 , comprising the second node and a network link connecting the second node to the first node, the second node being adapted:
to receive the timing packet with the updated timing information at the second node; and
to use the updated time information to synchronize the second node.
17. The system of claim 16 , wherein the second node comprises a local clock and is adapted to use the updated time information to synchronize the second node by using the updated time information to synchronize the local clock of the second node to a reference time source of the time server.
18. The system of claim 16 , wherein the second node is adapted to use the updated time information to synchronize the second node by determining a time epoch for the second node.
19. The system of claim 11 , wherein the first node is adapted to maintain a delay table, the delay table indicating respective delays associated with network links to which the first node is connected.
20. The system of claim 11 , wherein the first node is a packet network node and the second node is a base station of a mobile communications system connected to the packet network node.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/043,013 US20140029604A1 (en) | 2007-12-31 | 2013-10-01 | Communicating time information in a network to enable synchronization |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/006,151 US8559412B1 (en) | 2007-12-31 | 2007-12-31 | Communication time information in a network to enable synchronization |
US14/043,013 US20140029604A1 (en) | 2007-12-31 | 2013-10-01 | Communicating time information in a network to enable synchronization |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/006,151 Division US8559412B1 (en) | 2007-12-31 | 2007-12-31 | Communication time information in a network to enable synchronization |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140029604A1 true US20140029604A1 (en) | 2014-01-30 |
Family
ID=49321522
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/006,151 Expired - Fee Related US8559412B1 (en) | 2007-12-31 | 2007-12-31 | Communication time information in a network to enable synchronization |
US14/043,013 Abandoned US20140029604A1 (en) | 2007-12-31 | 2013-10-01 | Communicating time information in a network to enable synchronization |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/006,151 Expired - Fee Related US8559412B1 (en) | 2007-12-31 | 2007-12-31 | Communication time information in a network to enable synchronization |
Country Status (1)
Country | Link |
---|---|
US (2) | US8559412B1 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110274124A1 (en) * | 2008-10-24 | 2011-11-10 | Massimiliano Tosti | Method and Device for Packet Network Synchronization |
US20130265933A1 (en) * | 2012-04-06 | 2013-10-10 | Harris Corporation | Tdma wireless communications system and related methods |
US20140328356A1 (en) * | 2012-01-19 | 2014-11-06 | Abb Technology Ag | Data transmission over packet switched network |
US20150312350A1 (en) * | 2012-11-30 | 2015-10-29 | Metso Automation Oy | Multi-channel sensor measurement method and system |
US20170301217A1 (en) * | 2014-09-09 | 2017-10-19 | Tyco Fire & Security Gmbh | Modular wireless mass evacuation notification system |
US20180139034A1 (en) * | 2016-11-17 | 2018-05-17 | Accedian Networks Inc. | Virtualized clocks |
WO2018093811A1 (en) * | 2016-11-15 | 2018-05-24 | Autoliv Asp, Inc. | Systems and methods for synchronizing processor operations over a communications network |
US10609137B2 (en) | 2015-08-24 | 2020-03-31 | Microsoft Technology Licensing, Llc | Global logical timestamp |
US11110316B2 (en) | 2018-02-13 | 2021-09-07 | Coulter Ventures, Llc. | Assembly for weightlifting racks |
USD931389S1 (en) | 2018-07-11 | 2021-09-21 | Coulter Ventures, Llc. | J-cup |
US20220141790A1 (en) * | 2018-09-28 | 2022-05-05 | Intel Corporation | Technologies for managing internal time synchronization |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9667365B2 (en) | 2008-10-24 | 2017-05-30 | The Nielsen Company (Us), Llc | Methods and apparatus to perform audio watermarking and watermark detection and extraction |
CA3008502C (en) * | 2009-05-01 | 2020-11-10 | The Nielsen Company (Us), Llc | Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content |
CN102907021B (en) * | 2010-05-17 | 2016-06-01 | 瑞典爱立信有限公司 | Optimize timing and be grouped transmission |
PL2676389T3 (en) * | 2011-02-15 | 2017-02-28 | Telefonaktiebolaget Lm Ericsson (Publ) | Method of providing a path delay asymmetry for time synchronization between a master and a slave clock across a communciation network |
GB2525929B (en) | 2014-05-09 | 2016-08-10 | Imagination Tech Ltd | Time stamp replication within a wireless network |
US11776053B2 (en) * | 2014-09-07 | 2023-10-03 | Codrut Radu Radulescu | Synchronized exchange system |
CN106656393B (en) * | 2016-12-28 | 2018-10-19 | 深圳市英特瑞半导体科技有限公司 | Clock synchronizing method and device |
US11782880B2 (en) * | 2019-01-04 | 2023-10-10 | International Business Machines Corporation | Synchronizing log data within a cluster |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5566180A (en) | 1994-12-21 | 1996-10-15 | Hewlett-Packard Company | Method for recognizing events and synchronizing clocks |
US6731622B1 (en) * | 1998-05-01 | 2004-05-04 | Telefonaktiebolaget Lm Ericsson (Publ) | Multipath propagation delay determining means using periodically inserted pilot symbols |
US6370159B1 (en) | 1998-07-22 | 2002-04-09 | Agilent Technologies, Inc. | System application techniques using time synchronization |
US6278710B1 (en) | 1998-09-10 | 2001-08-21 | Agilent Technologies, Inc. | Enhancements to time synchronization in distributed systems |
US6252445B1 (en) | 1999-03-31 | 2001-06-26 | Agilent Technologies, Inc. | Method and apparatus for extending a resolution of a clock |
US6493539B1 (en) | 1999-07-28 | 2002-12-10 | Lucent Technologies Inc. | Providing an accurate timing source for locating the geographical position of a mobile |
US6512990B1 (en) | 2000-01-05 | 2003-01-28 | Agilent Technologies, Inc. | Distributed trigger node |
JP3993508B2 (en) * | 2002-12-02 | 2007-10-17 | 株式会社エヌ・ティ・ティ・ドコモ | Wireless access network system, wireless communication method, synchronization server, and node device |
US7450845B2 (en) * | 2002-12-11 | 2008-11-11 | Nortel Networks Limited | Expandable universal network |
US20070081514A1 (en) * | 2004-06-25 | 2007-04-12 | Yoshihiko Shirokura | Method of synchronizing time between base stations, timing master device, and base station |
GB2443868A (en) * | 2006-03-21 | 2008-05-21 | Zarlink Semiconductor Ltd | Synchronising slave clocks in non-symmetric packet networks |
-
2007
- 2007-12-31 US US12/006,151 patent/US8559412B1/en not_active Expired - Fee Related
-
2013
- 2013-10-01 US US14/043,013 patent/US20140029604A1/en not_active Abandoned
Cited By (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9100135B2 (en) * | 2008-10-24 | 2015-08-04 | Telefonaktiebolaget L M Ericsson (Publ) | Method and device for packet network synchronization |
US20110274124A1 (en) * | 2008-10-24 | 2011-11-10 | Massimiliano Tosti | Method and Device for Packet Network Synchronization |
US20140328356A1 (en) * | 2012-01-19 | 2014-11-06 | Abb Technology Ag | Data transmission over packet switched network |
US9537648B2 (en) * | 2012-01-19 | 2017-01-03 | Abb Schweiz Ag | Data transmission over packet switched network |
US20130265933A1 (en) * | 2012-04-06 | 2013-10-10 | Harris Corporation | Tdma wireless communications system and related methods |
US8804603B2 (en) * | 2012-04-06 | 2014-08-12 | Harris Corporation | TDMA wireless communications system and related methods |
US10021189B2 (en) * | 2012-11-30 | 2018-07-10 | Valmet Automation Oy | Multi-channel sensor measurement method and system |
US20150312350A1 (en) * | 2012-11-30 | 2015-10-29 | Metso Automation Oy | Multi-channel sensor measurement method and system |
US10966154B2 (en) | 2014-09-09 | 2021-03-30 | Johnson Controls Fire Protection LP | Master slave wireless fire alarm and mass notification system |
US20170301217A1 (en) * | 2014-09-09 | 2017-10-19 | Tyco Fire & Security Gmbh | Modular wireless mass evacuation notification system |
US10212664B2 (en) * | 2014-09-09 | 2019-02-19 | Tyco Fire & Security Gmbh | Modular wireless mass evacuation notification system |
US10470127B2 (en) | 2014-09-09 | 2019-11-05 | Johnson Controls Fire Protection LP | Master slave wireless fire alarm and mass notification system |
US10477477B2 (en) * | 2014-09-09 | 2019-11-12 | Johnson Controls Fire Protection LP | Modular wireless mass evacuation notification system |
US10555262B2 (en) | 2014-09-09 | 2020-02-04 | Johnson Controls Fire Protection LP | Modular wireless mass evacuation notification system |
US10609137B2 (en) | 2015-08-24 | 2020-03-31 | Microsoft Technology Licensing, Llc | Global logical timestamp |
WO2018093811A1 (en) * | 2016-11-15 | 2018-05-24 | Autoliv Asp, Inc. | Systems and methods for synchronizing processor operations over a communications network |
US10317511B2 (en) | 2016-11-15 | 2019-06-11 | Veoneer Us, Inc. | Systems and methods for synchronizing processor operations over a communications network |
US10721009B2 (en) * | 2016-11-17 | 2020-07-21 | Accedian Networks Inc. | Virtualized clocks |
US20180139034A1 (en) * | 2016-11-17 | 2018-05-17 | Accedian Networks Inc. | Virtualized clocks |
US11356189B2 (en) * | 2016-11-17 | 2022-06-07 | Accedian Networks Inc. | Virtualized clocks |
US11110316B2 (en) | 2018-02-13 | 2021-09-07 | Coulter Ventures, Llc. | Assembly for weightlifting racks |
USD931389S1 (en) | 2018-07-11 | 2021-09-21 | Coulter Ventures, Llc. | J-cup |
US20220141790A1 (en) * | 2018-09-28 | 2022-05-05 | Intel Corporation | Technologies for managing internal time synchronization |
US11553446B2 (en) * | 2018-09-28 | 2023-01-10 | Intel Corporation | Technologies for managing internal time synchronization |
US11856546B2 (en) | 2018-09-28 | 2023-12-26 | Intel Corporation | Technologies for managing internal time synchronization |
Also Published As
Publication number | Publication date |
---|---|
US8559412B1 (en) | 2013-10-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8559412B1 (en) | Communication time information in a network to enable synchronization | |
US9525502B2 (en) | Timing over packet demarcation entity | |
US8274999B2 (en) | Method and apparatus for analyzing and qualifying packet networks | |
US7590151B2 (en) | Method and apparatus for aligning time references when separated by an unreliable data packet network | |
US9673970B1 (en) | Methods and systems for estimating frequency synchronization accuracy | |
US7876791B2 (en) | Synchronizing apparatus and method in packet network | |
US8644348B2 (en) | Method for generating a robust timing correction in timing transfer systems | |
US11153838B1 (en) | Time synchronization enhancements for integrated 5G-TSN networking | |
US20160170440A1 (en) | Method and devices for time transfer using end-to-end transparent clocks | |
US20170288801A1 (en) | Methods and systems for estimating offset and skew using linear programming | |
US20150124844A1 (en) | Clock synchronization system, clock synchronization method, and storage medium whereupon clock synchronization program is stored | |
EP2333993A1 (en) | Automatic management of timestamp-based synchronisation protocols | |
CN112166565B (en) | Timing synchronization for cable networks | |
JP2000115210A (en) | Distribution system | |
US9860003B2 (en) | Method for providing time holdover in packet-based timing deployments | |
EP2530860A1 (en) | A method and apparatus for transporting time related information in a packet switched network | |
US20130215753A1 (en) | Non-intrusive method for synchronizing master and slave clocks of a packet-switched network, and associated synchronization devices | |
US20150163154A1 (en) | Method and devices for packet selection | |
KR20090071923A (en) | Method for synchronization using single timesync frame in bridged local area network and appratus thereof | |
Song et al. | Time synchronization performance analysis considering the frequency offset inside 5g-tsn network | |
CN115459871A (en) | Time synchronization method, network device and storage medium | |
EP3968562A1 (en) | Communication control device and communication control method | |
US20220026857A1 (en) | Time transmission correction device, time transmission system, and delay measurement method | |
EP3420655B1 (en) | Methods and systems for estimating frequency synchronization accuracy | |
EP3420666B1 (en) | Methods and systems for estimating skew |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: RPX CLEARINGHOUSE LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ROCKSTAR CONSORTIUM US LP;ROCKSTAR CONSORTIUM LLC;BOCKSTAR TECHNOLOGIES LLC;AND OTHERS;REEL/FRAME:034924/0779 Effective date: 20150128 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |