JP2003219385A - Video conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video conference system with a high efficiency in which a facility cost can be reduced. <P>SOLUTION: The video conference system suitable for a conference held by a plurality of participant units (100a to 100f) each outputting an individual data stream, includes: a central control unit (110) for receiving the individual data stream and selectively outputting part or all of the streams; a decoding unit (122) for separating the received individual stream into an individual audio signal and an individual video signal, mixing the separated individual audio signal and individual video signal to generate a mixed audio data stream and a mixed video data stream; and an encoding unit (124) for encoding the mixed audio data stream and the mixed video data stream to generate a mixed data stream. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates generally to video conferencing systems, and more particularly to video data stream control systems and methods for video conferencing systems.

[0002]

2. Description of the Related Art Spatial distances have been greatly reduced by the progress of science and technology related to communication technology. for that reason,
Message exchanges between people are no longer restricted to distances between regions. People have the custom of living together. In that case, mutual information exchange is important as a part of life. Meetings are one method used to solve common problems that exist within a group.

In a situation where communication is not sufficiently permeated, a meeting can be held only in a situation where all the participants of the meeting each go to a predetermined place. When communication technology and multimedia information technology have become popular, audio and video of conference participants are converted into digital signals by an electronic device and further into a data stream, and a local area network (L
Data streams can be sent and received via network technologies such as AN). The respective data streams, comprising audio and video, are simultaneously displayed on the display system of the conference participants, eg a personal computer system. In this case, the transmission and control of data is handled by a unified protocol. Thus, modern videoconferencing is not limited by the distance between regions.

The simple working principle of a video conference is illustrated in FIG. In FIG. 1, a broadcasting system such as a computer system (100a) is installed near the user (a). The computer system (100a) is provided with an acquisition device (104) for acquiring the video and audio of the user (a). Similarly, a computer system (100d) is installed near the user (d), and the acquisition device (104) is installed in the computer system (100d).
Is provided. In this case, each user has a corresponding broadcasting system (100a, 100b, ..., 100).
f). Each user sends a data stream to the control unit (102) via a local area network (LAN) using the same transmission protocol. The individual data streams are mixed to form a mixed data stream, which is then transmitted to each user's broadcast system. Generally, the control unit (10
2) is controlled by the chair of the conference and the number of inputs to the broadcasting system is controlled. For example, users (a, b,
The data of c, d) are selected and mixed, and the mixed data is
It is sent to all users (a, b, ..., F). The computer (100a) of the user (a) is displaying images of four users. Similarly, the computer (100d) of the user (d) is displaying images of the same four users. In this case, the display may or may not have a separate video. All other users are also viewing the video of user (a, b, c, d).

To hold a video conference by using the video conference system as shown in FIG. 1, a conventional system as shown in FIG. 2 is used. A plurality of users who are conference participants, such as (100a, 100b, ..., 100f), input individual data streams to the data stream control center (110). Data Stream Control Center (110)
Is controlled by the chair of the meeting. Data processing of video data requires a large amount of calculation power in the central processing unit (CPU), and other related members include
A large amount of processing power is required. Therefore, the load is limited. Generally, when displaying in real time, 4
Only one data stream can be selected.

The four selected data streams are input into a multipoint control unit (MCU) (112). Here, the multipoint control unit (11
2) has an audio / video (A / V) signal decoding / encoding processing unit (114). The individual data streams that are input are decoded into video and audio signals. The individual audio and video signals are then mixed and encoded into another mixed data stream. The mixed data stream is then sent to each unit (1
00a, 100b, ..., 100f) for broadcasting.

[0007]

In the video conferencing system described above, the decoding and coding of the data stream is handled by only one processing unit (114). When processing data, a large amount of CPU is used to decode and encode audio data and video data.
Calculation is required. Therefore, the processing unit (114)
Will handle a fairly large amount of load. Due to the limited computing power, the efficiency of the conventional video conferencing system is not ideal, and the equipment cost of the system cannot be successfully reduced.

[0008]

In order to solve the above-mentioned problems, the present invention provides a video conference system suitable for a conference held by a plurality of participant units. This video conference system includes a central control unit, a decoding unit, and an encoding unit. In this case, the central control unit receives the plurality of individual data streams from the plurality of participant units and selectively outputs all or part of the data streams. The decoding unit receives the selected plurality of individual data streams, decomposes the individual data streams into individual audio signals and individual video signals, and mixes the decomposed individual audio signals and the decomposed individual video signals. As a result, a mixed audio data stream and a mixed video data stream are formed. The encoding unit receives the mixed audio data stream and the mixed video data stream, forms the mixed data stream by encoding the mixed audio data stream and the mixed video data stream, and further forms the mixed data stream. To the participant unit.

The decoding process and the encoding process are processed by two separate processing units. As a result, in the video conference system according to the present invention, the calculation process is
The decoding chip and the coding chip can be performed separately from each other. The individual chips that make up these two chips do not have to have high computational power. Therefore, the circuit configuration of the chip, compared to the conventional system,
It will be much simpler. Along with this, the cost is reduced. Therefore, not only the efficiency of the video conference system is improved, but also the equipment cost of the system is reduced accordingly.

In addition, in the present invention, the encoding process and the decoding process are individually performed. This allows broadcast weighting to be dynamically incorporated into the computational operation for each data stream. Therefore, when the data stream is broadcast, several related individual data streams can be broadcast with weighting. As a result, the effect of the video conference can be emphasized and enhanced.

The present invention further provides a multi-unit data stream control system suitable for conferences conducted by multiple participant units. In this case, each participant unit outputs a separate data stream. This data stream control system comprises a decoding unit and an encoding unit. Here, the decoding unit receives the plurality of individual data streams, decomposes the individual data streams into individual audio signals and individual video signals, and mixes the decomposed individual audio signals and the decomposed individual video signals. As a result, a mixed audio data stream and a mixed video data stream are formed. The encoding unit receives the mixed audio data stream and the mixed video data stream, forms the mixed data stream by encoding the mixed audio data stream and the mixed video data stream, and further forms the mixed data stream. Is transmitted to the participant unit for broadcasting.

The above-mentioned decoding unit and coding unit can be realized by two processing chip units (or two video processing chip units).
When either one of the two processing chip units is dynamically selected as the decoding chip, the other chip becomes the coding unit.

The present invention further provides a data stream control method having a plurality of inputs. In this method, a decoding unit and an encoding unit are prepared; a plurality of data streams are input to the decoding unit; the decoding unit decomposes the data stream into individual audio signals and individual video signals, Mixing the decomposed individual audio signals and the decomposed individual video signals forms a mixed audio data stream and a mixed video data stream; an encoding unit combines the mixed audio data stream and the mixed video data stream. As a result, a mixed data stream for broadcasting is formed.

[0014]

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings are provided to provide a better understanding of the present invention and are incorporated as part of the specification. The accompanying drawings illustrate embodiments of the present invention and, in cooperation with the following description, serve to explain the principles of the invention.

FIG. 1 schematically shows the operating principle of a video conference according to the prior art.

FIG. 2 schematically shows a prior art video conference system by a block diagram.

FIG. 3 schematically shows, by a block diagram, a video conference system according to the present invention.

One of the main features of the invention is the use of a decoding unit and a coding unit which are separate and independent of each other. Therefore, the complexity of the processing chip is reduced. Therefore, the manufacturing cost is reduced and various broadcast weights can be applied to each broadcast window.

In order to explain the features of the present invention,
1 illustrates an embodiment. The present invention employs a data transmission protocol to transmit signals over a network for video conferencing. A data transmission protocol conforming to the communication standard is, for example, H.264. 320, H.M. 321,
H. 323, H.H. 324, ... etc. The data transmission protocol also has the function of adapting network types, video standards, audio standards, multi-task control, safety protocols, control protocols, modems, etc. to meet all the functional requirements of video conferencing protocols. There is. In the present invention, by changing the hardware configuration under the transmission protocol, the further function of improving the efficiency of the transmission protocol is fulfilled.

FIG. 3 is a block diagram schematically showing a video conference system according to the present invention. However, software protocols and interface standards are not shown. In FIG. 3, each conference participant
(For example, (100a, 100b,
, 100f) and an acquisition unit (104) adapted to the broadcast unit. Broadcast units are typically participant units (1
00a-100f) and is used by conference participants. The individual data streams generated by the participant units (100a-100f) are all via a data stream control center (1) via an input path, eg a local area network.
10) is input into.

Data Stream Control Center (110)
Is usually controlled by the chair of the meeting. Due to the limited display capabilities of the system, the individual data streams of some of the participants are dynamically selected and broadcast for each participant. On the other hand, the broadcast can be simultaneously transmitted to all the participants. Further, the data stream control center (110) may have a voice recognition capability capable of automatically recognizing the speaking participant, and may also automatically select a broadcast. As shown in FIG. 3, for example, four participants are selected and simultaneously broadcast. The individual data streams of the selected participants are input to the multipoint control unit (120) where they are subjected to a decoding process and an encoding process. By mixing the individual data streams, a mixed data stream according to the communication protocol is generated. The mixed data stream is output and sent to each participant unit (100a-100f) via the network for broadcasting.

In the prior art, the processing unit was overloaded, resulting in poor processing efficiency and high equipment costs. The present invention provides a method for performing a decoding process and an encoding process by using two processing chip units in the multipoint control unit (120).

Within the multipoint control unit (120), first of all, the selected individual data stream is a speech / audio signal known as a decoding unit (122).
It is input to the video (A / V) decoding unit (122). The decoding unit (122) decodes the individual data stream into an individual audio signal and an individual video signal.
A mixed audio data stream is formed by mixing a plurality of individual audio signals, and a mixed video data stream is formed by mixing a plurality of individual video signals. In the mixing process, various individual broadcast indexes, that is, various individual broadcast weights can be added to the corresponding individual audio signals or individual video signals according to physical requirements. For example, by giving a large weight to the current speaker, the ratio of the current speaker can be increased in the broadcast screen to make the current speaker stand out. Due to the fact that the decoding unit (122) does not perform the encoding process, the construction and manufacture of the processing chip for the decoding unit can be much simplified. Decoding unit (122)
Since the calculation load of is not so heavy, it is not necessary to use an expensive high speed chip as a processing chip for the decoding unit. The adjustment of broadcast weights does not necessarily have to be processed in the decoding unit (122), but can be processed by a further processing unit.

The mixed audio data stream and mixed video data stream output from the decoding unit (122) after the decoding process and the mixing process are sent to the encoding unit (124). Encoding unit (12
In 4), the mixed audio data stream and the mixed video data stream are mixed to form a mixed data stream according to the protocol. The mixed data stream is then output to each participant unit for broadcasting.

Since the encoding unit (124) performs only the encoding process, the workload is not so heavy. Therefore, it is not necessary to use an expensive high speed chip. In addition, if the broadcast weight setting is not processed in the decoding unit (122), the broadcast weight setting should be processed in the encoding unit (124). Which unit sets the broadcast weighting depends on the workload level of the decoding unit (122) and the coding unit (124).

In addition, the decoding unit and the encoding unit described above are realized by two processing chip units. When either one of these two processing chip units is dynamically selected as the decoding unit, the other processing chip unit is selected as the encoding unit. In other words, one of the two processing chip units can be dynamically selected to perform the decoding processing according to the calculation load at that time, and the other processing chip unit can be used for the encoding processing. Can be used. And those roles can be changed.

In the characteristic point according to the present invention, for example, N
In the case of video with the TSC 60Hz standard, the single full screen output is 30Hz, and the maximum delay of this system (when not considering the delay in decoding or encoding and the network delay) is about 1/30. Seconds. In addition, when using PAL50Hz, for example, the delay is only about 1/25 second by using two processing chip units for the decoding process and the encoding process. Using two chips to handle the computational load and getting an optimal price / performance ratio is one of the key features of the present invention.

Therefore, the present invention has at least the following features and advantages. 1. The decoding process and the encoding process are performed by different processing chips, which reduces the workload of the chip and reduces the equipment cost. 2. In the present invention, broadcast weights are assigned to participant units selected for broadcast. This allows the video of the speaker to stand out in some windows. 3. In the hardware configuration according to the present invention, the decoding process and the encoding process are performed by different processing chips and the conventional H.264 standard is used. It is possible to generate functions that cannot be obtained by the H.323 protocol. 4. The conferencing system according to the present invention is capable of handling multiple videoconferencing participants by using local area network transmission without spatial constraints. 5. The present invention has a function of automatically recognizing the current speaker, and can automatically select a broadcast. 6. The conferencing system according to the present invention is capable of selecting multiple participant units for broadcasting without the traditional constraint that only four can be selected at the same time. Since the decoding process and the encoding process are processed by different chips, the calculation capability can be improved.

Although the present invention has been described with reference to particular embodiments, without departing from the spirit of the invention,
It will be apparent to those skilled in the art that modifications can be made to the embodiments. Accordingly, the scope of the invention is defined by the claims, not the detailed description above.

[Brief description of drawings]

FIG. 1 is a diagram schematically showing an operating principle of a video conference according to a conventional technique.

FIG. 2 is a block diagram schematically showing a conventional video conference system.

FIG. 3 is a block diagram schematically showing a video conference system according to the present invention.

[Explanation of symbols]

100a participant unit 100b participant unit 100c participant unit 100d participant unit 100e participant unit 100f participant unit 110 data stream control center (central control unit) 120 multipoint control unit (multi-unit data stream control system) 122 decoding Unit 124 Encoding unit

Claims (4)

[Claims] 1. A video conferencing system suitable for a conference held by a plurality of participant units, each of which outputs an individual data stream, wherein the plurality of individual data streams are received and all or part of the data stream is received. A central control unit capable of selectively outputting a plurality of selected individual data streams, decomposing the individual data streams into individual audio signals and individual video signals, and decomposing these individual audio signals. A decoding unit for forming a mixed audio data stream and a mixed video data stream by mixing the signal and the decomposed individual video signals; receiving the mixed audio data stream and the mixed video data stream, and mixing them Audio data stream and mixed video data stream By encoding the arm, to form a mixed data stream,
And a coding unit for transmitting the mixed data stream to the participant unit for broadcasting. 2. A multi-unit data stream control system, suitable for a conference held by a plurality of participant units, each outputting an individual data stream, comprising a first processing chip unit and a second processing chip unit. A first processing chip unit and a second processing chip unit, wherein one of them is dynamically selected as a decoding unit, and the other is selected as an encoding unit; Mixed audio data by receiving a plurality of separate individual data streams, decomposing the individual data streams into individual audio signals and individual video signals, and mixing the decomposed individual audio signals and the decomposed individual video signals. A decoding unit forming a stream and a mixed video data stream; The audio data stream and the mixed video data stream are received, and the mixed audio data stream and the mixed video data stream are encoded to form a mixed data stream, and the mixed data stream is formed by the participant. An encoding unit for transmitting to the unit for broadcasting;
A multi-unit data stream control system comprising: 3. A multi-unit data stream control system suitable for a conference conducted by a plurality of participant units, each of which outputs a separate data stream, the system comprising: receiving a plurality of separate data streams; A decoding unit for decomposing an individual audio signal and an individual video signal, and mixing the decomposed individual audio signal and the decomposed individual video signal to form a mixed audio data stream and a mixed video data stream; A mixed audio data stream and the mixed video data stream are received, a mixed data stream is formed by encoding the mixed audio data stream and the mixed video data stream, and the mixed data stream is further joined. To the unit Multi-unit data stream control system characterized by comprising; a coding unit subjected to broadcast by Shin. 4. A method of controlling a data stream having a plurality of inputs, comprising: providing a decoding unit and an encoding unit; inputting a plurality of data streams to the decoding unit; The data stream
A mixed audio data stream and a mixed video data stream are formed by decomposing an individual audio signal and an individual video signal and mixing the decomposed individual audio signal and the decomposed individual video signal; A control method characterized by forming a mixed data stream for broadcasting by combining the mixed audio data stream and the mixed video data stream.