JP2007005894A - Data transmission apparatus and control method thereof, and control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a data transmission apparatus capable of executing data transmission with excellent real time performance by preventing occurrence of underflow even when a delay is caused in message transfer on a network. <P>SOLUTION: A message processing section 104 always monitors a transmission buffer 103, acquires voice data from the transmission buffer 103 in a timing when the voice data are stored up to a prescribed threshold, embeds the voice data to an HTTP request and transmits the HTTP request to a server apparatus. In this case, a transfer time measurement section 105 starts measuring time to measure a time until a response is received from the server apparatus. A transmission data amount determining section 106 executes statistics processing of the measured times and judges the tendency of permanent variations (not temporary) in times required for transmission/reception of an HTTP message to revise the threshold of the transmission buffer 103. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a data transmission apparatus that transmits continuous data such as voice over a network such as the Internet, a control method thereof, and a control program for realizing the control method.

  In recent years, data distribution systems that require real-time performance, such as live distribution systems and video conference systems using an IP (Internet Protocol) network, have been proposed (see, for example, Patent Document 1).

  In general, when performing real-time (real-time) voice communication, the transmission side transmits voice data by dividing it into small pieces in order to reduce the delay amount due to voice communication. Furthermore, it is necessary to transmit the audio data divided into small parts at regular intervals. This is to prevent underflow as will be described later on the receiving side. When an underflow occurs on the receiving side, sound interruption or echo occurs during reproduction, which makes the user uncomfortable.

Consider performing real-time voice communication from a client device to a server device using HTTP (HyperText Transfer Protocol). First, the client device transmits an HTTP message including voice data in the HTTP request to the server device, and receives a response from the server device. By repeating transmission and reception of this HTTP message, continuous voice communication from the client device to the server device becomes possible. However, as described above, in order to perform voice communication with high real-time characteristics, the client device needs to perform the transmission / reception processing of the HTTP message in as small a unit as possible at regular intervals.
JP 2002-330489 A

  However, in the HTTP message transmission / reception process, in an actual network environment, it is difficult to implement the HTTP message transmission / reception unit in small units and at regular intervals due to various delay factors on the apparatus or the network.

  Specifically, in the network path between the client device and the server device, when passing through the proxy server device, the transfer process is delayed by about several hundred milliseconds by temporarily storing the HTTP message, or the HTTP header is rewritten. The HTTP session may be disconnected after transferring the HTTP response. Also, if anti-virus software is running on the client device, it may cause the same phenomenon as when using a proxy server device. In addition, transmission / reception processing of HTTP messages may be delayed due to various causes such as packet drop on the network.

  Due to these factors, HTTP message transmission / reception processing is delayed, and the server device on the audio playback side underflows, causing sound interruptions and echoes during playback, causing discomfort to the user. .

  In view of the above-described conventional problems, the present invention provides a data transmission apparatus capable of preventing underflow and performing data transmission with excellent real-time performance even when a delay occurs in message transfer on the network, and a control method thereof, An object of the present invention is to provide a control program.

  In order to achieve the above object, the present invention provides a data transmission apparatus for transmitting data to an external apparatus in real time via a network, a buffering means for temporarily storing data input from the input apparatus, and the buffering means Request message creating means for creating a request message including the data stored in the message, message transmitting / receiving means for transmitting the created request message to the external device, and receiving a response message for the request message from the external device, and Included in the request message based on the message transfer time measured by the transfer time measuring means, the transfer time measuring means for measuring the message transfer time from the transmission of the request message until the response message is received Characterized by comprising a data amount determination means for determining the amount of data.

  According to another aspect of the present invention, there is provided a data transmission apparatus control method for transmitting data to an external device in real time via a network, a step of temporarily storing input continuous data in the buffering means, and the buffering means A request message creating step for creating a request message including the stored data, a message transmitting / receiving step for sending the request message to the external device, and receiving a response message to the request message from the external device, and sending the request message A transfer time measuring step for measuring a message transfer time from when the response message is received to the response message, and a data amount of data included in the request message based on the message transfer time measured in the transfer time measuring step Decision and executes the data amount determining step for.

  The present invention also provides a computer-readable control program for executing a control method of a data transmission device that transmits data to an external device in real time via a network, and temporarily inputs input continuous data. Storing in the buffering means, a request message creating step for creating a request message including the continuous data stored in the buffering means, and transmitting the request message to the external device, from the external device to the request message A message transmission / reception step for receiving a response message, a transfer time measurement step for measuring a message transfer time from transmission of the request message to reception of the response message, and a transfer time measurement step. Based on the serial message transfer time, and having a data amount determining step of determining the amount of data included in the request message.

  According to the present invention, underflow can be prevented by using the transmission / reception time of a request message and a response message even if a delay occurs in message transfer on the network, and the quality is excellent in real time. Data transmission can be performed.

  Embodiments of a data transmission apparatus, a control method thereof, and a control program according to the present invention will be described with reference to the drawings.

<Device configuration>
FIG. 1 is a block diagram showing the main functions of an audio transmission system provided with an audio communication apparatus which is a data transmission apparatus of the present invention.

  This audio transmission system is a system for transmitting audio data input from the audio input device 100 from the client device 101 to the server device 200 via the network 108 in real time. The client apparatus 101 includes an audio data processing unit 102, a transmission buffer 103, a message processing unit 104, a transfer time measurement unit 105, a transmission data amount determination unit 106, a network processing unit 107, and the like. The voice transmission system shown in FIG. 1 includes a client device 101 and a server device 200 as communication terminal devices, but may have a configuration of client devices or a configuration of server devices.

  The audio data processing unit 102 receives an audio signal from the audio input device 100, performs A / D conversion processing and audio compression processing, and the transmission buffer 103 temporarily stores the compressed audio data. The message processing unit 104 creates an HTTP request message that includes the audio data stored in the transmission buffer 103.

  The transfer time measuring unit 105 measures the time (message transfer time) from when the created HTTP request is transmitted to when the HTTP response returned from the server device 200 is received. Statistics of message transfer time are collected to determine the amount of voice data included in the HTTP request. The network processing unit 107 transmits and receives an HTTP request and an HTTP response.

  Although the voice input device 100 may exist in the client device 101, there is no significant difference as a functional block constituting the voice transmission system, and therefore this embodiment will be described as a separate device.

On the other hand, the server device 200 includes a network processing unit 201, a reception buffer 202, a message processing unit 203, and an audio data processing unit 204. The network processing unit 201 transmits / receives an HTTP response or an HTTP request. The reception buffer 202
The received voice data is accumulated, and the message processing unit 203 creates an HTTP response. The audio data processing unit 204 performs D / A conversion while sequentially expanding the audio data.

  FIG. 2 is a block diagram illustrating an example of a hardware configuration of the client apparatus 101 illustrated in FIG.

  The client apparatus 101 includes an audio data processing unit 102 and a transmission buffer 103, a CPU 901, a ROM 902, a RAM 903, an input device controller (KBC) 905, a CRT controller (CRTC) 906, and a disk controller shown in FIG. A (DKC) 907 and a network interface controller 908 are connected via a system bus 904 so as to communicate with each other.

  The CPU 901 comprehensively controls each component connected to the system bus 904 by executing software stored in the ROM 902 or the HD 911. That is, the CPU 901 reads out a processing program according to a predetermined processing sequence from the ROM 902 or the HD 911 and executes it, whereby the message processing unit 104, the transfer time measurement unit 105, and the transmission data amount determination unit 106 shown in FIG. In addition to realizing each function corresponding to, control for realizing the operation according to the present embodiment (see FIG. 3) is performed. The RAM 903 functions as a main memory or work area for the CPU 901.

  An input device controller (KBC) 905 controls instruction inputs from a mouse 909a and a keyboard 909b which are input devices, and a CRT controller (CRTC) 906 controls display on a CRT display (CRT) 910. A disk controller (DKC) 907 controls an access operation to the hard disk (HD) 911, and a network interface controller (NIC, corresponding to the network processing unit 107 in FIG. 1) 908 performs connection processing with the network 108.

<Operation according to the present embodiment>
Next, the operation according to the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the client apparatus 101 of this embodiment.

  When voice communication from the client device 101 to the server device 200 is started, first, the voice data processing unit 102 performs A / D conversion on a voice signal input from the voice input device 100 in real time, and further performs voice compression. To the transmission buffer 103 (step S11). Since audio signals are continuously input, they are successively stored in the transmission buffer 103 in a short period of several milliseconds to several tens of milliseconds.

  The message processing unit 104 constantly monitors the transmission buffer 103, and when the audio data is accumulated up to a predetermined threshold (described later) in the transmission buffer 103 (step S12), acquires the audio data from the transmission buffer 103 (step S12). S13), embedded in the HTTP request (step S14). Then, the message processing unit 104 instructs the network processing unit 107 through the transfer time measuring unit 105 to transmit an HTTP request in which the audio data is embedded. At this time, the transfer time measuring unit 105 starts time measurement using a timer (step S15). The network processing unit 107 transmits the HTTP request instructed from the message processing unit 104 to the server device 200 via the network 108.

  On the server device 200 side, when an HTTP request is received, the audio data is accumulated in the reception buffer 202 and an HTTP response is created. Then, the message is returned to the client apparatus 101 via the network 108. Thereafter, the audio data stored in the reception buffer 202 is sequentially decompressed, further D / A converted, output to the audio output device 205, and the audio is reproduced.

  In the client device 101, the network processing unit 107 receives the HTTP response and sends it to the message processing unit 104 through the transfer time measuring unit 105. Upon confirming the HTTP response, the transfer time measuring unit 105 stops the operating timer (step S16), calculates the time (message transfer time) taken from the transmission of the HTTP request to the reception of the HTTP response, and the transmission data The amount determining unit 106 is notified (step S17).

  The transmission data amount determination unit 106 performs statistical processing of the message transfer time measured from the past to the present. Then, the amount of transmission data included in one HTTP request is determined by comparing the message transfer time measured this time with the past measurement time (step S18). Specifically, if it is determined that the message transfer time measured this time is longer than the past measurement time, the threshold value of the capacity of transmission data stored in the transmission buffer 103 being monitored is set to the message processing unit 104. An instruction to increase the amount of transmission data to be transferred at a time is issued. Also, if it is determined that the message transfer time measured this time is shorter than the past measurement time, the threshold of the capacity of transmission data stored in the transmission buffer 103 monitored for the message processing unit 104 is reduced, An instruction to reduce the transmission data capacity to be transferred at a time is issued. (Step S20).

  For example, the transmission data amount determination unit 106 determines that the message transfer time is longer or shorter than the past measurement time. For example, the average measurement time of the last 10 seconds is the previous 10 seconds. There is a method of observing a constant fluctuation tendency such that the average measurement time is twice or half of the average measurement time. In this case, the threshold value is not changed many times in a short time by detecting a temporary delay of the network. If the threshold value is changed many times in a short time, the average communication time during reproduction in the server device 200 fluctuates, and sound interruptions are likely to occur.

  In addition, the first threshold value (initial value) of the transmission buffer 103 determined by the transmission data amount determination unit 106 is preferably determined in consideration of a delay factor in the actual network configuration. For example, an HTTP request is sent to the server device 200 via the proxy server device, or a delay factor in the network configuration such as whether antivirus software is used on the client device 101 is checked, and these are used. If the transmission buffer 103 is not used, the transmission buffer 103 is set to a small threshold. At this time, it is effective to use a plurality of thresholds depending on the actual environment.

  In this way, an initial value optimum for the network configuration for communication can be used as the threshold value of the transmission buffer 103. Furthermore, by taking statistics of measurement time from the past, it is possible to dynamically change the amount of data that is transferred as voice data at once, so even in a network environment where the message transmission / reception interval takes longer than expected, it is optimal. Network communication can be performed with a threshold value. Even after the voice communication is terminated, the threshold value of the transmission buffer 103 is held for each server device to be communicated and reused as an initial value for the next voice communication. Stable voice communication is possible.

  After receiving the HTTP response on the client apparatus 101 side, the message processing unit 104, when the voice data stored in the transmission buffer 103 exceeds the threshold again, sends the voice data from the transmission buffer 103 for the next HTTP request. , HTTP data in which the audio data is embedded is created and transmitted to the server apparatus 200 in the above-described procedure. At this time, if the HTTP response is not received even if the audio data exceeds the threshold, the next HTTP request transmission process waits until the reception of the HTTP response.

<Changes in the amount of buffer storage in the receive buffer>
Next, transition of the buffer accumulation amount of the reception buffer 202 in the server device 200 will be described with reference to FIGS. 4A, 4 </ b> B, and 4 </ b> C are waveform diagrams for explaining the transition of the buffer accumulation amount of the reception buffer in the server apparatus 200.

(A) Transition of buffer accumulation amount in an ideal state The message transfer time, which is the time from when an HTTP request is transmitted to when an HTTP response is received, is necessary until the voice data exceeds the threshold of the transmission buffer 103. When the time is shorter than the time, the buffer storage amount of the reception buffer 202 in the server device 200 changes as shown in FIG.

  In the example of FIG. 4A, at time t200 when an HTTP request including audio data is received, the remaining buffer accumulation amount b200 is temporarily acquired up to the buffer accumulation amount b201 by acquiring the audio data included in the HTTP request. Increase. After that, it is shown that the buffer accumulation amount is gradually reduced by reproducing the sound until time t201 when the next HTTP request is received. In the present embodiment, it is assumed that processing for returning an HTTP response to the client apparatus 101 in response to completion of reception of the HTTP request is performed. Then, the process of receiving the next HTTP request at time t202, accumulating it in the reception buffer 202 and reproducing it is repeated.

  In the example of FIG. 4A, it is assumed that communication is performed in an ideal state. Specifically, since the server apparatus 200 returns an HTTP response to the client apparatus 101 in response to the completion of reception of the HTTP request, the client apparatus 101 can receive the HTTP response without delay. For this reason, the average of the buffer accumulation amount is also constant, and there is no underflow in which there is no audio data in the reception buffer 202.

(B) Transition of buffer accumulation amount when this embodiment is not applied Next, a transition of the buffer accumulation amount of the reception buffer 202 in the server apparatus 200 when this embodiment is not applied will be described.

  Server when client device 101 cannot send HTTP requests at regular intervals due to delay factors on the network, such as when using a proxy server device on the network path or using specific anti-virus software The transition of the buffer accumulation amount of the reception buffer 202 of the device 200 is shown in FIG.

  The server device 200 returns an HTTP response at time t300. However, the client apparatus 101 cannot receive an HTTP response even at time t301 due to a delay on the network. For this reason, the client apparatus 101 cannot naturally transmit the next HTTP request.

  Since the client apparatus 101 has not received an HTTP request at time t301, which is an ideal HTTP request reception timing, the amount of audio data stored in the reception buffer 202 of the server apparatus 200 becomes zero at time t302. . That is, underflow occurs, and sound interruption occurs in the audio output device 205.

  Thereafter, the client apparatus 101 transmits the next HTTP request in response to the reception of the HTTP response, but the server apparatus 200 receives the HTTP request at time t303. At this time, the buffer storage amount is temporarily recovered and the audio reproduction is started. However, since it takes time to arrive at the next HTTP request, a buffer underflow occurs again, and the audio output device 205 It becomes inaudible as a continuous sound.

(C) Transition of buffer accumulation amount when this embodiment is applied Next, the buffer accumulation amount in server apparatus 200 when this embodiment is applied under the same conditions as in the example of FIG. The transition of is shown in FIG.

  When a proxy server device is used on the network path or specific anti-virus software is used as a delay factor, and when the client device 101 knows the delay factor as connection information In addition, the client apparatus 101 sets, for example, twice the amount of audio data included in one HTTP request as shown in FIG.

  As a result, in the reception buffer 202 of the server device 200, the audio data doubles at time t400 and increases to the buffer accumulation amount b401. For this reason, even if the next HTTP request is received at twice the message interval thereafter, the buffer accumulation amount b400 is reduced, so that occurrence of buffer underflow can be prevented.

  In the present embodiment, not only the correspondence to the network configuration known in advance as described above, but also, for example, when the network route through which the HTTP message passes is changed in the middle or the network load suddenly increases, etc. Even when a typical network delay occurs, it can be dealt with accurately. That is, as described above, by measuring the message transfer time and taking statistics, the network delay is detected, and the voice data transmitted together with the HTTP message is increased to prevent buffer underflow in the server device 200. can do.

<Advantages of this embodiment>
Ideally, in order to perform voice communication with high real-time characteristics, it is important for the client apparatus 101 to perform HTTP message transmission / reception processing in as small a unit as possible at regular intervals. However, the HTTP message transmission / reception process may be delayed in the network path between the client apparatus 101 and the server apparatus 200. As the cause, for example, there is a delay due to the network configuration such as via a proxy server device or anti-virus software running on the client device 101. In addition, if the network path through which the HTTP message passes is changed in the middle, or the packet drop on the network or the network load suddenly increases, the delay due to the temporary network environment is complicated and entangled. It may appear to the device that network delays are constantly occurring.

  In such a case, transmission / reception of an HTTP message between the client apparatus 101 and the server apparatus 200 (an HTTP message including voice data in the HTTP request is transmitted to the server apparatus and a response is received from the server apparatus) is adversely affected. That is, the time from the transmission of the HTTP message to the reception of the response becomes longer than the transmission data interval (the amount of audio data included in one request).

  In such a tendency, in the present embodiment, adjustment is made to increase the amount of data to be transmitted at a time to increase the transmission interval. That is, the amount of voice data included in the HTTP request is set to a data amount equal to or more than the constant message transmission / reception time in the network configuration / environment at that time. Specifically, the message processing unit 104 constantly monitors the transmission buffer 103, acquires the voice data from the transmission buffer 103 at a timing when the voice data is accumulated up to a predetermined threshold, and embeds the voice data in the HTTP request. Send to. At this time, time measurement is started by the transfer time measuring unit 105, and the time until a response is received from the server device is measured. The transmission data amount determination unit 106 performs the statistical processing of the measurement time, determines a constant fluctuation tendency (not temporary) of the time required for transmission / reception of the HTTP message, and changes the threshold value of the transmission buffer 103. .

  In such a configuration of the present embodiment, for example, the delay factor of the actual network configuration is known in advance such as sending an HTTP request via the proxy server device or using anti-virus software on the client device 101. In this case, the threshold value of the transmission buffer 103 is determined in consideration of these delay factors. This makes it possible to use an initial value optimum for the network configuration for communication as the threshold value. In addition to adapting to network configurations known in advance, even in a network environment where constant network delays such as those described above occur, it is dynamically optimized by taking statistics of measurement times from the past. It can be changed to a different threshold value.

  As a result, the amount of audio data included in the HTTP request can be set to an optimum unit for the network environment to be used, and the HTTP request transmission interval can be made constant, so that a buffer underflow occurs on the server device 200 side. This makes it possible to prevent sound interruptions and unpleasant echoes.

  Note that an object of the present invention is to supply a storage medium recording a program code of software that implements the functions of the embodiment to a system or apparatus, and a computer (or CPU, MPU, etc.) of the system or apparatus as the storage medium. This can also be achieved by reading and executing the stored program code.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code and the storage medium storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a floppy (registered trademark) disk, a hard disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, and a DVD. -RW, DVD + RW, magnetic tape, nonvolatile memory card, ROM, etc. can be used. Alternatively, the program code may be downloaded via a network.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. A case where part or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. This includes the case where the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  In this case, the program is supplied by downloading directly from a storage medium storing the program or from another computer or database (not shown) connected to the Internet, a commercial network, a local area network, or the like.

It is a block diagram which shows the principal part function of an audio | voice transmission system. It is a block diagram which shows an example of the hardware constitutions of a client apparatus. It is a flowchart which shows operation | movement of embodiment. It is a wave form diagram which shows transition of the buffer accumulation amount of a reception buffer.

Explanation of symbols

100 voice input device 101 client device 102 voice data processing unit 103 transmission buffer 104 message processing unit 105 transfer time measurement unit 106 transmission data amount determination unit 107 network processing unit 108 network 201 server device 202 reception buffer 205 voice output device 901 CPU
902 ROM

Claims (6)

In a data transmission device that transmits data to an external device in real time via a network,
Buffering means for temporarily storing data input from the input device;
Request message creating means for creating a request message including data stored in the buffering means;
A message transmitting / receiving means for transmitting the created request message to the external device and receiving a response message to the request message from the external device;
A transfer time measuring means for measuring a message transfer time from transmission of the request message to reception of the response message;
A data transmission apparatus comprising: a data amount determining unit that determines a data amount of data included in the request message based on the message transfer time measured by the transfer time measuring unit. The data amount determining means includes
When it is determined by the transfer time measuring means that the message transfer time is increasing, the amount of data included in one of the request messages is increased, and when it is determined that the message transfer time is decreasing The data transmission apparatus according to claim 1, wherein the data amount is reduced.   3. The data transmission apparatus according to claim 2, wherein the data amount is not changed when the increase / decrease of the message transfer time is temporary or when the range of the increase / decrease is small.   The data according to any one of claims 1 to 3, wherein when the data amount data specified by the data amount determining means is stored in the buffering means, the next request message is transmitted. Transmission equipment. In a control method of a data transmission device that transmits data to an external device in real time via a network,
Temporarily storing the input continuous data in the buffering means;
A request message creating step for creating a request message including data stored in the buffering means;
A message transmission / reception step of transmitting the request message to the external device and receiving a response message to the request message from the external device;
A transfer time measuring step of measuring a message transfer time from transmission of the request message to reception of the response message;
A method for controlling a data transmission device, comprising: performing a data amount determination step of determining a data amount of data included in the request message based on the message transfer time measured in the transfer time measurement step. A computer-readable control program for executing a data transmission device control method for transmitting data to an external device in real time via a network,
Temporarily storing the input continuous data in the buffering means;
A request message creating step for creating a request message including continuous data stored in the buffering means;
A message transmission / reception step of transmitting the request message to the external device and receiving a response message to the request message from the external device;
A transfer time measuring step for measuring a message transfer time from transmission of the request message to reception of the response message;
A control program comprising: a data amount determining step for determining a data amount of data included in the request message based on the message transfer time measured in the transfer time measuring step.

Images