JP2002164915A - System and method for synchronizing communications - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system and method that allows a receiver side to synchronously receive communication data that are separately transmitted. SOLUTION: A sender divides communication data to be sent and transmits the divided communication data through separate paths. The receiver side synchronizes the communication division data sent through the separate paths and reproduces the data. In order to allow the receiver side to synchronously receive the communication division data, a 'channel number', a 'transmission time', a 'data generating time', 'presence/absence of voice information', and 'presence/absence of video information' or the like are used as attached information to the division communication data. Then the receiver side synchronizes the communication division data that are independently transmitted. Or the transmitter side transmits the communication division data synchronized with each other in advance to the receiver side, and the receiver side synchronizes the received communication division data by using synchronization information used by the transmitter side when the transmitter side transmits the communication division data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a network, such as the Internet, which is not reliable and has variable communication delays. An object of the present invention is to provide a communication system and a communication method which can be applied to a use field in which communication data is transferred and a certain synchronization control is required on a receiving side.

[0002]

2. Description of the Related Art In the present specification, the description will be made using the Internet system for the sake of concrete description, but the present invention is not limited to the Internet.

A related patent (Japanese Patent Application No. 2000-015504)
No., Japanese Patent Application No. 2000-38514, Japanese Patent Application No. 2000-04
No. 9775) has shown a method of realizing the transfer of communication data having a steady flow rate such as video data using a network having no reliability such as the Internet and having a variable communication delay. According to the system of the related patent, it is possible to control the communication delay between the sender and the receiver with respect to a series of communication data with a constant accuracy based on the periodic structure of the communication data.

FIG. 22 is a diagram for explaining a problem that occurs in practical application of the prior art. In practical applications, there is a need to transfer not only a series of communication data but also a plurality of mutually associated communication data. Specifically, since a series of communication data requires an extremely wide band, when it cannot be transferred using one path of the network (FIG. 22A), the communication data is divided into a plurality of communication data, When communication data is communicated using different network paths, and synchronization is integrated on the receiving side (FIG. 22B),
When it is desired to transfer image data from a plurality of video sources such as a multi-channel image such as a stereoscopic image, or to synchronize images and sounds from a plurality of points on a receiving side in a network conference (FIG. 22C). is there.

In a related patent, a transmitting side adds a sequence number and an identification code which is a characteristic of data to a series of communication data (Digital Video data) and transfers the data. The method of detecting redundant transmission or loss of data and taking appropriate measures was described. However, this method does not provide the following means, which causes a problem in practical application. A) A method of transferring a series of communication data through different routes. B) A method in which a plurality of communication data are associated and synchronized on the receiving side.

In the following description, as a specific example, Digita
l Video (DV) data is taken as an example, but the scope of the present invention is not limited to DV. FIG. 23 shows a general structure of the "relay device" referred to in the related patent. Although this relay device can originally have a transmission / reception function, in the description of the present invention, transmission / reception is described separately for clarity of description.

[0007] When the relay apparatus 10 receives the DV data from the transmission side, it receives the DV data at the IEEE1394 adapter 11 and terminates the line. Then, the received DV data is input to the DV → DV / IP conversion circuit 12, the DV data is converted into a data format necessary for sending to the Internet, and input to the Internet adapter 14. Internet adapter is converted DV
Send data to the Internet.

Conversely, when an IP packet containing DV data is received by the Internet adapter 14 from the Internet, the DV / IP → DV conversion circuit 13
And convert the IP packet to DV data to IEEE13
94 Input to the adapter 11. This DV data is transmitted to the terminal that is the transmission destination, and the DV data is displayed on the terminal.

FIG. 24 is a diagram for explaining the problems of the prior art. For example, in a case where the image data is multiplexed into two channels and transmitted to the Internet, and the receiving side separates the two-channel image data and views the image data of each channel, the sender of the video channel 1 and the video channel The image data from the sender 2 is transmitted to the relay device 20 on the sender side.
, It is necessary to combine channel 1 and channel 2 and transmit the result to the receiving-side relay device 21 via the Internet. However, in this case, in order for the Internet to transfer image data for two channels, about 70
A bandwidth of ８０80 Mbps is required. However, it is difficult to secure such a wide band, and according to the Internet protocol, the image data for these two channels is automatically divided into a plurality of pieces and transferred within the Internet. Will be combined and then broken down into each channel.

[0010]

The following problems exist in the technology of the above-mentioned related patent. A) It is necessary to divide one communication into a plurality of communications in accordance with the available bandwidth of the network, and to transfer the communications as independent communications through different routes. B) It is necessary to delay-adjust and synchronize data transferred from one sender with a plurality of different communication delays on the receiving side. C) It is necessary to delay-adjust and synchronize data transferred from a plurality of senders with a plurality of different communication delays on the receiving side.

An object of the present invention is to provide a communication system and a communication method capable of receiving a plurality of communications synchronously on a receiving side.

[0012]

According to the present invention, there is provided a system for performing communication using a plurality of network paths having different communication delays, wherein additional information for divided communication is added via the plurality of network paths. And transmitting means for transmitting the divided communication data based on the additional information, and receiving the data by synchronizing and receiving the plurality of divided communication data.

According to the present invention, communication data requiring synchronization control can be transmitted using a plurality of network paths having different communication delays, so that large-capacity communication data such as moving images and stereoscopic images is transmitted. In this case, even if a single network path does not have enough bandwidth, the receiving side can receive such communication data while ensuring synchronization.

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a diagram for explaining DV division communication. Here, the division communication means that one communication data is originally transmitted as a plurality of communication data, and the reception side collects the communication data into one communication data. Specifically, in order to combine two-channel stereoscopic image data (each channel is DV data) with one relay device and transfer it as one communication data on the Internet, about 70 to 80 is required.
Mbps bandwidth is required. Between the sender and the recipient,
It suffices if there is a sufficient communication band for performing such communication, but otherwise communication cannot be performed.

In general, there are a plurality of routes connecting a sender and a receiver on a network. If the total bandwidth of each of the plurality of routes exceeds a required communication band, one communication is performed by a plurality of routes. The communication can be realized by dividing the communication into communication and distributing the communication to different routes.

FIG. 2 is a diagram showing a transmission repeater according to an embodiment of the present invention. In the related patent, a frame number, a sequence number, and the like are provided as identifiers, and the receiving side performs communication data recognition processing by using these identifiers. In the embodiment of the present invention, in order to realize divided communication, "channel number", "transmission time", "data generation time", "presence / absence of voice information", and "presence / absence of image information" ”, Etc., an identifier for identifying data is newly added. On the receiving side, the data is synchronized and integrated using the identifier and the identifier indicated in the related patent.

The DV data of channels 1 and 2 input to the IEEE1394 adapter 101 are synthesized, and DV → DV /
In the IP conversion device 102, the DV data format is mapped to the IP format. Then, in the communication data dividing device 103, the DV data is divided by the following method, and each is transmitted to the Internet via the Internet adapters 104-1 and 104-2.

The communication data dividing device 103 divides the communication data into a plurality of communication data and performs processing for adding the identifier. As a method of dividing the communication data, for example, in the case of Digital Video data, various methods can be considered as follows. A) Divide image and audio B) Divide by frame number such as odd frame and even frame C) Divide by channel number D) Divide by sequence number E) Divide by time information such as transmission time and data generation time Any division is possible as long as it can be identified by an identifier as in

As a method of specifying a different route for each divisional communication on the transmitting side, a packet is transmitted by specifying a source route also in the Internet, and a multi-home (multiple addresses are assigned) is allocated to the receiver side. There are known techniques such as Embodiments of the present invention are based on these techniques.

FIG. 3 is an explanatory diagram of a receiving repeater for explaining a communication delay adjusting process of a single sender. Receiving relay device 1
14, the Internet adapter 110-1,
In 110-2, upon receiving data divided and transmitted from the Internet, the communication data integrating device 111
, The divided communication data is integrated into one,
The / IP → DV converter 112 extracts DV data from the DV data mapped to the IP format, and transmits the DV data to the DV data display devices 115-1 and 115-2 via the IEEE1394 adapter 113. At this time, the DV data display devices 115-1 and 115-2 extract the respective channels from the DV data.

In the split communication, since a single sender can arbitrarily add an identifier on a predetermined basis, the receiving side can adjust the communication delay by referring to the identifier. This can be realized essentially by adding a communication data integration device 111 having a communication data integration function to the configuration of the related patent.

The communication data integrating device 111 is a single queue such as a frame buffer shown in the related patent, and uses the identifier added in the embodiment of the present invention in addition to the method shown in the related patent to order the communication data. , Missing, redundant inspection.

FIG. 4 is a diagram for explaining a communication delay adjusting function of a plurality of senders. In a case where a plurality of senders respectively create and transmit communication data, the following means can be considered as a method of integrating the plurality of communication data on the receiving side. A) An identifier is added in the same manner as when a single sender sends data with a certain synchronization between the senders.

For example, in FIG. 4, the sender 120-1 receiving the transmission data of the video channel 1 and the sender 120-2 receiving the transmission data of the video channel 2 exchange synchronization information with each other to transmit data mutually. Are transmitted in synchronization with each other.

As a means for synchronizing the senders, it is conceivable to notify the related patents and the identifier added in the embodiment of the present invention to each other. For example, if a sequence number that does not overlap is used by notifying the sequence number, a unique sequence number is obtained on the receiving side, so that integration processing can be performed.

FIG. 5 is a diagram showing a configuration of a transmission relay apparatus for transmitting data in synchronization with a transmission side using a sequence number. IEEE1394 adapters 125-1 and 125-that independently receive DV data of channels 1 and 2, respectively.
2 is DV → DV / I in each transmission relay device.
In the P converters 126-1 and 126-2, the DV data is mapped to the IP format, and the identifier adding unit 1
In 27-1 and 127-2, a sequence number is added. At this time, the synchronizers 128-1 and 128-2 notify each other of the Sea Kenshi numbers used by each other so that they do not use the same sequence number. Thus, the data to which the sequence number has been added is transmitted from the Internet adapters 129-1 and 129-2 to the Internet.

FIG. 6 is a diagram for explaining an apparatus for synchronizing and integrating communication data of a plurality of senders on the receiving side. As another method of integrating a plurality of communication data on the receiving side, there is the following method. B) The sender creates and transmits communication data completely independently,
Integrate multiple communication data with a single receiver.

In a multipoint video conference, the senders may be located far apart and there is no way for the senders to synchronize. In such a case, it is necessary to synchronize only on the receiving side. For example, if the sender is "data sending time"
And the "data generation time" are stored in the identifier and sent, and the receiving side can synchronize by comparing this time. GPS (Global Positioning System)
If the clocks of multiple senders are synchronized, synchronization can be achieved by comparing the absolute times. Even when the absolute time cannot be obtained, the difference between the time (local time) of the sender 1 and the time of the sender 2 (local time) is constant. Therefore, the time can be synchronized by designating the time of the difference to the receiver. .

FIG. 7 is a diagram for explaining another configuration for synchronizing and integrating a plurality of communication data from a plurality of senders on the receiving side.
As another method of integrating a plurality of communication data on the receiving side, there is the following method. C) When there are a plurality of recipients, the sender creates and transmits communication data completely independently, and integrates the plurality of recipients by synchronizing with each other.

In this case, the independent receivers notify each other of the identifier of the communication data being processed and synchronize with each other. For example, synchronization can be achieved when audio data is generated from a state where there is no audio data. This utilizes an empirical rule that the timing at which an image starts generally coincides with the transition from a state where no audio data is present to a state where audio data is generated. Similarly, modifications such as synchronization when image data is generated from a state where there is no image data and synchronization with a fixed image / audio data pattern are also possible.

FIG. 8 is a diagram for explaining the configuration of the receiving repeater in the case of a plurality of senders. Communication data input from the Internet to the respective Internet adapters 130-1 and 130-2 are transmitted to the synchronization units 131-1 and 131-.
2, the synchronization is established between the receiving repeaters and the DV /
In the IP → DV converters 133-1 and 133-2,
DV data is extracted from the IP format. In the synchronization units 131-1 and 131-2, the synchronization device 132-
1 and 132-2 notify each other of the synchronization timing, and the synchronization units 131-1 and 131-2 synchronize. Synchronized communication data converted to DV data is IEEE
In the 1394 adapters 134-1 and 134-2, the data is transmitted to the display terminal as DV data.

FIG. 9 is a diagram for explaining a synchronization configuration in the case of a plurality of senders and a plurality of receivers. As another method of integrating a plurality of communication data on the receiving side, there is the following method. D) When there are a plurality of recipients, a communication data synchronization mechanism is provided outside the recipients, which adjusts and synchronizes each of the plurality of recipients.

For example, a video display can be observed by a human and synchronized by setting a communication delay for each recipient. According to the related patent, if a communication delay is set, the delay is maintained thereafter. Therefore, if a communication delay is initially set for a plurality of recipients, synchronization is maintained thereafter. Various modifications of the means using such an external synchronization device are conceivable.

FIG. 10 is a diagram for explaining the configuration of a receiving repeater when a synchronizer is used. Each of the display devices 140-140 receives the DV data from the receiving relay device independently.
1 and 140-2 each perform a synchronization operation using the synchronization device 141 in the above-described manner, send the result to the synchronization unit of the reception relay device, and synchronize independent DV data. .

In the division communication according to the embodiment of the present invention, the broadband communication is divided into a plurality of paths, and the total bandwidth of the plurality of paths is set to be greater than the band required by the original broadband communication, thereby achieving stable communication. To achieve. However,
The effective available bandwidth of the path dynamically changes under the influence of other communication sharing the communication link constituting the path. As a result, a band that should have been sufficient halfway may suddenly run short, or conversely, a route that should have had no room may be vacant.

FIG. 11 and FIG. 12 are diagrams for explaining the effect on the divided communication due to the dynamic change of the used bandwidth. For example, in the case shown in FIG. 11 (initial state), more divided data should flow on the path A than on the path B. However, when the state of the network changes and FIG. Data should flow. In other words, if the division ratio can be changed according to the state of the network (effective available bandwidth),
Stable communication can be realized.

Here, since data communication can be divided into arbitrary ratios by using a frame number or a sequence number, it is only necessary to be able to dynamically grasp the effective bandwidth for each path. In general, it is difficult to directly measure the effective available bandwidth, but estimation can be performed by providing a network information collection device and monitoring the network state to collect information. As a result, if the total estimated effective available bandwidth is less than the required bandwidth, the number of divisions is increased and data delivery is performed via another new route.

FIG. 13 is a diagram for explaining a transmission relay apparatus that dynamically changes the division method using network information. Channels 1 and 2 received from two DV displays
DV data is received by the IEEE1394 adapter 150 and becomes one DV data. Next, DV → DV / I
In the P converter 151, the DV data is mapped to the IP format. The DV data mapped to the IP format is transmitted to the communication data dividing device 15.
3 and is divided using the additional information described above.
At this time, the network information collecting device 152 obtains the used bandwidth information of the network from the management information of the network, and notifies the communication data dividing device 153 of the information. The communication data dividing device 153 divides the communication data based on the band information obtained from the network information collecting device 152, by deciding to which channel the band is to be allocated more. The divided communication data is transmitted from the Internet adapters 154-1 and 154-2 to the Internet.

As the network information that can be used for bandwidth estimation, various methods are conceivable, such as A) the packet loss rate of the route, B) the amount of packet data passing through the network interface on the transmitting side. In A), for example, 5
If the packet loss rate is 20% in a state where 0 Mbps (divided) broadband data is flowing through one specific path, the effective available bandwidth is 50 × (1-0.2) = 40M
bps. As a method of measuring the packet loss rate, as shown in a related patent, there is a means of measuring a packet loss rate using a sequence number attached to synchronization data at a receiver and feeding back information to a transmission side. In B), the target (divided) broadband communication checks the amount of data passing through the transmitting-side network interface, and regards it as an effective available band. In this method, when the network interface unit is in a bottleneck or overflow state, the effective available bandwidth can be accurately estimated, but otherwise, the accuracy is lower than A). However, on the Internet, dropped packets are said to occur on the link closest to the sending or receiving host,
It can be expected that almost accurate estimates can be obtained with this method. In addition, in A), information exchange between the receiving side and the transmitting side is necessary to acquire information, whereas in B), there is an effect that information can be obtained only inside the transmitting side.

In the above-mentioned dynamic division communication, it is desirable to reflect the network information acquired promptly in the division policy. Conversely, if there is no significant change in the network information, the change is reflected in the division policy. In some cases, only overhead increases and performance is degraded.

For example, the originally required bandwidth of broadband communication is 100 Mbps, and the effective available bandwidth (or an estimated value thereof) when data communication is divided into two paths is 9
It is assumed that vibration has occurred between 9 and 101 Mbps. At this time, when the effective available bandwidth becomes 99 Mbps, the bandwidth becomes insufficient and a new alternative path is searched for, and data is flowed using that. However, when the effective available bandwidth becomes 101 Mbps, the use of the third path is stopped. Occurs, and frequent route searches must be performed. By preventing small changes in network information from being reflected in the data division policy, this phenomenon can be avoided.

FIG. 14 is a diagram for explaining a configuration of a transmission relay apparatus for preventing a small change in network information from being reflected in a data division policy. First, a network information evaluation unit 165 that stores the network information obtained one time ago and quantifies the difference from the newly obtained network information, and a network information determination unit that determines whether the difference exceeds a certain threshold value 166 are provided. And
The instruction from the network information collecting device 164 to the communication data dividing device 162 is masked according to the output result of the network information determining unit 166. As a method of digitizing the network information, it is conceivable to use the numerical value of the effective bandwidth obtained from the network information as it is. Then, when the change in the effective use bandwidth is equal to or greater than the threshold,
Change the data division method. At this time,
When the effective use bandwidth becomes smaller than the required bandwidth, the data division scheme may be changed sensitively, and the change in the data division scheme when the bandwidth becomes large may become slow. That is, if the bandwidth is reduced, communication data cannot be completely transmitted, and the communication quality is degraded. Since the quality does not deteriorate, the change of the data division method is slowed down. This is because the network information determination unit 166
Can be realized by changing the threshold value between when the bandwidth increases and when the bandwidth decreases.

In an unreliable network such as the Internet, the state of the network may deteriorate for a moment and return to a stable state immediately thereafter. That is, since the network information collecting device 164 picks up this momentary change in the network state and responds to it, the performance deteriorates. Therefore, a method of changing the division policy only when the network state has changed for a certain long period of time is adopted. It may be performed.

This can be realized as follows. First,
A network information evaluation unit 165 that stores network information used as a basis for determining a current communication data division policy and quantifies a difference from newly acquired network information, and a network information evaluation unit 165 that exceeds a certain threshold. A network information determination unit 166 is provided for determining whether or not the network information has been detected. And
The network information determination unit 166 masks the instruction from the network information collection device 164 to the communication data division device so as not to be transmitted in principle, and exceeds the threshold for a plurality of consecutive times predetermined by the network information determination unit 166. Only when the mask is released. At the same time, the reference network information is updated.

FIG. 15 is a diagram showing a detailed configuration of the data communication system. The two relay devices 30 and 32 are connected to the IE
An EE1394 adapter 35 is provided, and the first network or the second
Connected to a network.

Each of the relay devices 30 and 32 has an Internet adapter 39, and is connected to the Internet via the Internet adapter 35.
The DV / IP transmitting unit 31 illustrated in FIG. 15 includes a capsule forming unit 37 that IP-encapsulates the IEEE1394 packet received via the IEEE1394 adapter 35 by a predetermined method, and a packet that is IP-encapsulated by the capsule forming unit 37. A packet redundantly transmitting unit 38 which receives and partially transmits packets to be transmitted to the Internet adapter 39 by a predetermined method.

FIG. 16 shows a detailed configuration of a 1-division communication data block of the communication data division DV / IP transmission unit 31. The encapsulation unit 31 shown in FIG. 16 has an Ethernet (registered trademark) packet format, an IP packet format,
A function of encapsulating an IEEE1394 packet using the DP packet format is provided. The header adding unit 54 adds the additional information received from the additional information creating unit 53 to the IEEE1394 packet as shown in FIG. It has a function of forming a DV / IP payload and adding an Internet header consisting of a 20-byte IP header, an 8-byte UDP header, and a 14-byte Etherframe header.

The frame detecting section 51 shown in FIG.
Is a series of image and audio information for one frame.
It has a function to detect the beginning of the IEEE1394 packet,
The counter 52 has a function of performing a coefficient operation in accordance with the detection result of the frame detection unit 51 and the input of the IEEE1394 packet.

Based on the count value of the counter 52, the additional information creating unit 53 generates additional information (including a frame number indicating the image frame to which the IEEE1394 packet belongs and a block number indicating the position where the corresponding IEEE1394 packet is located in the frame). (See FIG. 17).

On the other hand, the packet redundant transmission section 38 shown in FIG.
, The packet extraction / holding unit 56
In response to the instruction from 7, some of the packets formed by the above-described capsule forming unit 31 are transmitted in duplicate.
A target packet is extracted, and a packet extraction / holding unit 5
6 is provided.

Further, the redundantly transmitted packet insertion unit 58 shown in FIG. 16 reads a packet to be redundantly transmitted from the packet extraction / holding unit 56 in accordance with a predetermined procedure in accordance with an instruction from the redundant transmission control unit 57, and It has a function of inserting it into a series of packets input to the Internet adapter 39.

On the other hand, in the relay device 32 shown in FIG. 15, the packet input to the DV / IP receiving unit 36 via the Internet adapter 39 is
Into the additional information described above and the IEEE1394 packet.

The additional information separated from the DV / IP packet in this manner is analyzed by the additional information analysis unit 45, and the write control unit 43 operates in accordance with the analysis result, whereby the additional information is included in each IEEE1394 packet. DI
Each of the F blocks is written to the corresponding storage location in the reception buffer 42.

Further, the integrity judgment section 46 shown in FIG.
Receives the additional information from the capsule disassembly unit 44, collects information indicating the integrity of the digital video data belonging to each frame based on the additional information, as described later, and based on the collected information. , Determine the completeness of the specified frame, and
The data is restored by the data restoration unit 48.

On the other hand, the delay adjusting section 47 shown in FIG.
It operates according to the restoration instruction from the DV transmission control unit 49, and based on the additional information received from the capsule disassembly unit 44,
A frame to be restored is determined, and information designating this frame is transmitted to the integrity determining unit 46 and the DV data restoring unit 4.
8 has a function of sending a restoration instruction.

The DV data restoring unit 4 shown in FIG.
Digital video data restored by IEEE
Via the adapter 35, the packet stream is transmitted to the second network as an IEEE1394 synchronous mode packet stream, and is input to the digital video deck 50.

FIG. 18 is a diagram showing a detailed configuration of one communication data block of the communication data integration DV / IP receiver. As shown in FIG. 18, additional information obtained by decomposing a DV / IP packet by a capsule decomposing unit 44 is input to an additional information analyzing unit 45, and is included in an IEEE1394 packet corresponding to the additional information. The respective DIF blocks (DV data) are input to the write control unit 43.

The reception buffer 42 has a capacity corresponding to digital video data for k frames, and a storage area corresponding to each frame has a capacity corresponding to DV data included in an IEEE1394 packet. It consists of 250 storage locations corresponding to block numbers.

The write control unit 43 has a function of receiving the frame number and the block number from the additional information analyzing unit 45 and obtaining a storage location indicated by the information. The data is input to the reception buffer 42 together with the DV data.

On the other hand, the integrity judging section 46 shown in FIG.
Is provided with a counter control unit 71, a block counter unit 72, and an integrity evaluation unit 73.
2 performs a counting operation in accordance with an instruction from the counter control unit 71, and the result of the counting is provided to processing of the integrity evaluation unit 73.

The counter control unit 71 receives a code indicating the data type included as the additional information of the newly received packet from the additional information analysis unit 45, and performs a counting operation by the block counter unit 72 based on the code. It has a function to control

The block counter 72 includes, for example,
For k frames, the total number RBCi of received DV data blocks (i = 0 to k) and the number H of header blocks
BCi (i = 0 to k), the number of audio blocks ABCi (i
= 0 to k) and the number of video blocks VBCi (i = 0 to
Each of the storage areas has a function of operating a numerical value of the corresponding storage area in accordance with an instruction from the counter control unit 71.

In FIG. 18, the DV data restoring section 48 includes a block reading section 77 for reading DV data from the reception buffer 42 and a block reading section 77 for reading the DV data.
A synthesizing buffer 80 for holding the DV data read out by the above and the synthesizing data described below, a DV data organizing unit 79 for restoring the DV data using the data held in the synthesizing buffer 80, A part 78.

To the synthesizing control unit 78, a restoration instruction from the DV transmission control unit 49 shown in FIG. 15, an evaluation result by the above-described integrity evaluation unit 73, and a frame instruction to be described later are input. Based on these instructions and information,
The synthesis control unit 78 has a function of controlling the operations of the block reading unit 77, the synthesis buffer 80, and the DV data organization unit 79.

The delay adjusting unit 47 shown in FIG.
An arrival information updating unit 74 for updating arrival information indicating the most recently arrived DV data based on an analysis result received from the additional information analysis unit 45, and a delay amount holding unit for holding an appropriate number of delay frames DF as a transmission delay amount. K formed based on the DV data held in the reception buffer 42 based on the arrival information and the number of delayed frames DF described above.
And a frame determining unit 75 for selecting a frame to be restored from the plurality of frames.

FIGS. 19 and 20 are diagrams for explaining the data format. One frame, which is a basic element of digital video data, includes one frame of image data and audio data, and is composed of ten DIF sequences as shown by numbers following FIG. Have been.

This DIF sequence consists of 150 DIs.
As shown in FIG. 19B, 15 pieces of image information (v) are connected to one piece of audio information (a) following six pieces of frame header information (f), as shown in FIG. It has a structure where a regular pattern is repeated.

Each DIF block is shown in FIG.
As shown in the figure, 77 bytes of data are added with identification information represented by 3 bytes. The data structure shown in FIG. 19 represents the logical format of digital video data. When data is actually transferred over a network, a format according to the characteristics of the network is adopted.

For example, digital video data is
When transferring via a local bus conforming to the standard, FIG.
As shown in FIG. 0 (a), predetermined headers (CIP (Common Isochronous Packet) header and synchronous (ISO) header) and CRC are added to the six DIF blocks, and IEEE1 is added.
394 packets are formed, and a transfer operation is performed using the IEEE1394 packet as a transfer unit.

In this case, each DIF sequence corresponds to FIG.
As shown in FIG. 9 (b), separated by a vertical bar, 25
Since the video data is divided into IEEE1394 packets, one frame of digital video data contains 25 frames as shown in FIG.
It is transferred as zero IEEE1394 packets.

Therefore, the combination of DIF blocks included in each IEEE1394 packet has a regularity as shown in FIG. The repetition pattern indicated by an arrow in FIG.
DIF containing audio information in IEEE1394 packets
Block (a) and DIF block (v) containing image information
And the change pattern of the combination
The IEEE1394 packet corresponds to a part of the DIF sequence indicated by hatching in FIG.

DIF in the above DIF sequence
As is clear from the regularity of the arrangement of the blocks, similar repetition patterns are shown in FIG.
, And appear in the IEEE1394 packets indicated by numbers 17 to 24.

Further, the IEEE 1394 standard specifies that 8000 IEEE 1394 packets are transferred per second in the synchronous transfer mode, whereby video and audio devices such as a display device and a speaker connected to the local bus are connected. Through this, it is possible to reproduce 30 frames of image and audio information per second.

Therefore, by using the gateway connected to both the local bus conforming to the IEEE1394 standard and another type of network such as the Internet, the local bus conforming to the IEEE1394 standard can be used.
A second network similar to the first network connected to each other can be connected via a third network of another type, and the above-described IEEE1394 packets can be mutually exchanged.

FIG. 21 is a diagram for explaining a configuration for receiving a plurality of data on the receiving side in synchronization with the additional information. FIG. 21 illustrates a configuration for synchronizing a plurality of data using the additional information in the receiving-side relay device 202. The sender 200 divides communication data into a plurality of pieces of data, adds additional information to each piece of data, and transmits the data to the Internet 201. These divided data are input to the receiving-side relay device 202 following a plurality of network paths of the Internet 201. The relay device 202 extracts the additional information from the received data and analyzes what the additional information is. Then, the control unit 205 causes the write control unit 204 to control the writing of data to the reception buffer 205 according to the content of the additional information. For example, when the additional information is a “channel number”, control is performed so that data is allocated to different buffers for each channel.
When the additional information is “transmission time” and “data generation time”, data having the same time set as the additional information is input to the reception buffer 205 at the same timing and output from the reception buffer 205 at the same timing. Let it. When the additional information is “presence / absence of audio information” and “presence / absence of image information”, the reception buffer 205 of the data packet in which the information of the presence or absence of the audio information or the image information changes
And the output timing from the reception buffer 205 is controlled to be aligned.

(Supplementary Note 1) A system for performing communication using a plurality of network paths having different communication delays, wherein a plurality of divided communication data to which additional information for divided communication is added is transmitted via the plurality of network paths. And a receiving unit that integrates and synchronizes a plurality of divided communication data based on the additional information, and receives the divided communication data. (Supplementary note 2) The network is the Internet,
The system according to claim 1, wherein the communication data is image data.

(Supplementary Note 3) The transmitting means is provided for each of a plurality of senders, and the senders exchange information regarding the additional information with each other, and add the plurality of senders to communication data transmitted by each sender. 2. The system according to claim 1, wherein additional information having a uniform regularity is added to the entire sender and transmitted.

(Supplementary Note 4) The transmitting means is provided by each of a plurality of senders, each of the transmitting means independently transmits communication data, and the plurality of receivers having the receiving means are:
The system according to claim 1, wherein communication data is received synchronously by exchanging information about the additional information with each other.

(Supplementary Note 5) The transmitting means has a plurality of senders, each of which independently transmits communication data, and a plurality of receivers having the receiving means,
2. The system according to claim 1, further comprising a synchronization unit that adjusts a synchronization state by observing a reproduction state of the received data, and that performs synchronization by the synchronization unit to ensure synchronization of communication data.

(Supplementary note 6) The system according to Supplementary note 1, wherein the additional information is any one of a channel number of communication data, a transmission time of communication data, and a generation time of communication data.

(Supplementary Note 7) The system according to Supplementary Note 1, wherein the additional information uses information indicating the presence or absence of a specific data such as voice or image or a specific pattern.
(Supplementary note 8) The system according to supplementary note 1, wherein the transmission unit divides the communication data into a plurality of divided data and transmits the divided data using a plurality of network paths.

(Supplementary note 9) The system according to supplementary note 8, wherein the division is dynamically performed based on network information. (Supplementary note 10) Supplementary note 9 wherein the network information is continuously collected two or more times, and it is determined whether or not to change the division method by comparing the collected at least two pieces of network information. System.

(Supplementary note 11) The system according to supplementary note 10, wherein the difference in the network information is quantified, and when the difference is equal to or greater than a threshold, the division method is changed. (Supplementary note 12) The system according to supplementary note 9, wherein a packet loss rate of a communication path or a packet data amount passing through each network interface is used as the network information.

(Supplementary Note 13) A transmitting apparatus in a system for performing communication using a plurality of network paths having different communication delays, the transmitting apparatus comprising:
A transmission device, comprising: transmission means for transmitting a plurality of pieces of divided communication data to which additional information including information necessary for synchronization on a receiving side is added for divided communication.

(Supplementary Note 14) A system receiving apparatus that performs communication using a plurality of network paths having different communication delays, and integrates a plurality of pieces of divided communication data based on additional information necessary for a transmitting side to synchronize. Receiving means for receiving data in a synchronized manner.

(Supplementary Note 15) A method for performing communication using a plurality of network paths having different communication delays, wherein a plurality of pieces of divided communication data to which additional information for divided communication is added are transmitted via the plurality of network paths. (Supplementary Note 16) wherein the network is the Internet, and the receiving step is to integrate and synchronize the plurality of divided communication data based on the additional information. 16. The method according to supplementary note 15, wherein the communication data is image data.

(Supplementary Note 17) The transmitting step is performed by a plurality of senders, respectively, and the senders exchange information regarding the additional information with each other to transmit the plurality of transmission data to the communication data transmitted by each sender. 16. The method according to supplementary note 15, wherein additional information having a uniform regularity is added to the whole user and transmitted.

(Supplementary Note 18) The transmitting step is performed by a plurality of senders, respectively. The senders independently transmit communication data, and the plurality of receivers performing the receiving step mutually transmit the additional information. The method according to claim 15, wherein the communication data is received synchronously by exchanging information about the communication data.

(Supplementary Note 19) The transmitting step is performed by a plurality of senders, each of the senders independently transmits communication data, and the plurality of receivers having the receiving means reproduce the received data. The method according to claim 15, further comprising a synchronization step of adjusting a synchronization state by observing a state, and ensuring synchronization of communication data by performing adjustment by the synchronization means.

(Supplementary note 20) The method according to Supplementary note 15, wherein the additional information is any one of a channel number of communication data, a transmission time of communication data, and a generation time of communication data.

(Supplementary note 21) The method according to supplementary note 15, wherein the additional information uses information indicating the presence or absence of a specific data such as voice or image or a specific pattern. (Supplementary note 22) The method according to supplementary note 15, wherein in the transmitting step, the communication data is divided into a plurality of pieces of divided data and transmitted using a plurality of network paths.

(Supplementary note 23) The method according to supplementary note 22, wherein the division is dynamically performed based on network information. (Supplementary note 24) Supplementary note 23, wherein the network information is continuously collected two or more times, and whether the division method is changed is determined by comparing the collected at least two pieces of network information. The method described in.

(Supplementary note 25) The method according to supplementary note 24, wherein the difference of the network information is quantified, and when the numerical value is equal to or larger than a threshold, the division method is changed. (Supplementary note 26) The method according to supplementary note 23, wherein a packet loss rate of a communication path or an amount of packet data passing through each network interface is used as the network information.

[0094]

According to the present invention, when the bandwidth is insufficient on one path of the network, the communication data is divided and transmitted, sent to the receiving side using a plurality of paths, and divided on the receiving side. It is possible to combine and use the communication data sent.

In particular, when there are a plurality of senders, a plurality of communication data sent from the plurality of senders can be properly received by synchronizing on the receiving side.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating DV division communication.

FIG. 2 is a diagram illustrating a transmission relay device according to an embodiment of the present invention.

FIG. 3 is an explanatory diagram of a reception relay device for explaining communication delay adjustment processing of a single sender;

FIG. 4 is a diagram illustrating a communication delay adjustment function of a plurality of senders.

FIG. 5 is a diagram illustrating a configuration of a transmission relay device that transmits data in synchronization with a transmission side using a sequence number.

FIG. 6 is a diagram illustrating an apparatus for synchronizing and integrating communication data of a plurality of senders on the receiving side.

FIG. 7 is a diagram illustrating another configuration for synchronizing and integrating a plurality of communication data from a plurality of senders on a receiving side.

FIG. 8 is a diagram illustrating a configuration of a reception relay device in the case of a plurality of senders.

FIG. 9 is a diagram illustrating a synchronization configuration in the case of a plurality of senders and a plurality of recipients.

FIG. 10 is a diagram illustrating a configuration of a reception relay device when a synchronization device is used.

FIG. 11 is a diagram (part 1) for explaining an influence on divided communication due to a dynamic change of an execution use band.

FIG. 12 is a diagram (part 2) for explaining the influence on the divided communication due to the dynamic change of the used bandwidth.

FIG. 13 is a diagram illustrating a transmission relay device that dynamically changes a division method using network information.

FIG. 14 is a diagram illustrating a configuration of a transmission relay apparatus for preventing a small change in network information from being reflected in a data division policy.

FIG. 15 is a diagram showing a detailed configuration of a data communication system.

FIG. 16 shows a communication data division DV / IP transmission section 31-1.
FIG. 3 is a diagram illustrating a detailed configuration of a divided communication data block.

FIG. 17 is a diagram illustrating additional information.

FIG. 18 is a diagram showing a detailed configuration of one communication data block of the communication data integration DV / IP receiving unit.

FIG. 19 is a view for explaining a data format (part 1);
It is.

FIG. 20 illustrates a data format (part 2).
It is.

FIG. 21 is a diagram illustrating a configuration for receiving a plurality of data in synchronization on the receiving side based on additional information.

FIG. 22 is a diagram illustrating a problem that occurs in actual application of the related art.

FIG. 23 is a diagram illustrating a general structure of a relay device.

FIG. 24 is a diagram for explaining a problem of the related art.

[Explanation of symbols]

30, 32 relay device 31 DV / IP transmission unit 33 DV / IP reception unit 34 digital video camera 35 IEEE1394 adapter 36 DV / IP reception unit 37 capsule formation unit 38 packet redundant transmission unit 39 Internet adapter 41 DV / IP transmission unit 42 reception Buffer 43 Write control unit 44 Capsule disassembly unit 45 Additional information analysis unit 46 Integrity determination unit 47 Delay adjustment unit 48 DV data restoration unit 49 DV transmission control unit 50 Digital video deck 51 Frame detection unit 52 Counter 53 Additional information creation unit 54 Header Addition unit 56 Packet extraction / holding unit 57 Redundant transmission control unit 58 Redundant transmission packet insertion unit 60 DV / IP transmission unit 71 Counter control unit 72 Block counter unit 73 Integrity evaluation unit 74 Arrival information update unit 75 Frame determination unit 76 Delay amount Holder 77 Read unit 78 synthesis control unit 79 DV data organization unit 80 synthesis buffer 100 transmission relay device 101 IEEE1394 adapter 102 DV → DV / IP conversion device 103 communication data division device 104-1, 104-2 internet adapter 110-1, 110-2 Internet adapter 111 Communication data integration device 112 DV / IP → DV conversion device 113 IEEE1394 adapter 114 Reception relay device 125-1, 125-2 IEEE1394 adapter 126-1, 126-2 DV → DV / IP conversion device 127- 1, 127-2 Identifier adding unit 128-1, 128-2 Synchronizing device 129-1, 129-2 Internet adapter 130-1, 130-2 Internet adapter 131-1, 131-2 Synchronizing unit 132-1, 132- 2 Synchronizer 133- 133-2 DV → DV / IP conversion device 134-1, 134-2 IEEE1394 adapter 150 IEEE1394 adapter 151 DV → DV / IP conversion device 152 Network information collection device 153 Communication data division device 154-1, 154-2 Internet adapter 162 Communication data division device 164 Network information collection device 165 Network information evaluation unit 166 Network information judgment unit 200 Sender 201 Internet 202 Relay device 203 Additional information analysis unit 204 Write control unit 205 Reception buffer

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C064 BB05 BC11 BC16 BC20 BD01 BD08 5K030 GA03 GA13 HA08 HB02 HC01 HD05 JA05 JL07 LB06

Claims (20)

[Claims] 1. A system for performing communication using a plurality of network paths having different communication delays, comprising: transmitting means for transmitting, via the plurality of networks, a plurality of divided communication data to which additional information for divided communication is added. And a receiving means for integrating and synchronizing and receiving a plurality of divided communication data based on the additional information. 2. The system according to claim 1, wherein said network is the Internet, and said communication data is image data. 3. The transmitting means has a plurality of senders, respectively, and the senders exchange information regarding the additional information with each other, and add the plurality of senders to communication data transmitted by each sender. 2. The system according to claim 1, wherein additional information having a uniform regularity is added and transmitted. 4. The transmitting means includes a plurality of senders, each of which independently transmits communication data, and a plurality of receivers having the receiving means relate to each other with respect to the additional information. The system of claim 1, wherein communication information is received synchronously by exchanging information. 5. The transmission means includes a plurality of senders, each of which independently transmits communication data, and wherein a plurality of receivers having the reception means are in a reproduction state of the reception data. The system according to claim 1, further comprising: a synchronization unit that adjusts a synchronization state by observing the synchronization data, and that the communication unit performs synchronization by using the synchronization unit to secure synchronization of communication data. 6. The system according to claim 1, wherein the additional information is any one of a channel number of communication data, a transmission time of communication data, and a generation time of communication data. 7. The system according to claim 1, wherein the additional information uses information indicating the presence or absence of a specific data such as voice or image or a specific pattern. 8. The system according to claim 1, wherein said transmitting means divides the communication data into a plurality of divided data and transmits the divided data using a plurality of network paths. 9. The system according to claim 8, wherein the division is performed dynamically based on network information. 10. The method according to claim 1, wherein the network information is continuously collected two or more times, and whether or not the division method is changed is determined by comparing the collected at least two pieces of network information. Item 10. The system according to Item 9. 11. The system according to claim 10, wherein the difference of the network information is converted into a numerical value, and when the numerical value is equal to or larger than a threshold, the division method is changed. 12. The system according to claim 9, wherein a packet loss rate of a communication path or a packet data amount passing through each network interface is used as said network information. 13. A transmitting apparatus in a system for performing communication using a plurality of network paths having different communication delays, wherein the transmitting apparatus is required to synchronize divisional communication on a receiving side via the plurality of network paths. A transmission device, comprising: transmission means for transmitting a plurality of divided communication data to which additional information including information is added. 14. A system receiving apparatus for performing communication using a plurality of network paths having different communication delays, wherein a plurality of divided communication data are integrated on the basis of additional information required for synchronization added by a transmitting side. Receiving means for receiving data in a synchronized manner. 15. A method for performing communication using a plurality of network paths having different communication delays, comprising: transmitting a plurality of divided communication data to which additional information for divided communication is added via the plurality of network paths. And a receiving step of integrating and synchronizing and receiving a plurality of divided communication data based on the additional information. 16. The transmitting step is performed by a plurality of senders, respectively, and the senders exchange information regarding the additional information with each other, and add the plurality of senders to the communication data transmitted by each sender. The method according to claim 15, wherein additional information having a uniform regularity is added and transmitted. 17. The transmitting step is performed by a plurality of senders, each of which independently transmits communication data, and the plurality of receivers performing the receiving step mutually transmit information regarding the additional information. 16. The method according to claim 15, wherein the communication data is received synchronously by exchanging. 18. The transmitting step is performed by a plurality of senders, each of the senders independently transmits communication data, and the plurality of receivers having the receiving means check a reproduction state of the received data. The method according to claim 15, further comprising a synchronization step of adjusting a synchronization state by observing, and ensuring synchronization of communication data by performing adjustment by the synchronization means. 19. The method according to claim 15, wherein said transmitting step divides the communication data into a plurality of divided data and transmits the divided data using a plurality of network paths. 20. The method according to claim 19, wherein the division is performed dynamically based on network information.