JPH07162831A - Control method for video conference communication equipment - Google Patents

Abstract

PURPOSE:To suitably perform video conference communication while referring to a small picture area by displaying the small area picture in any part of an image displayed on an entire display screen so as not to exert any influence even when it is hidden. CONSTITUTION:This video conference communication equipment is provided with an in-picture picture display function for displaying small pictures over the display screen of a video monitoring device 18. This slave picture is displayed at any one of display positions PL1-PL4 set at the four positions of a master picture, for example. The display position of the slave picture is basically designated by a user but this equipment is provided with an operation mode for automatically moving the slave picture to any position so as not to hide the valid part of an image displayed on the master screen as much as possible. This image synthesizing processing is executed by a display control part 17. Then, a picture superimposing the slave picture on the master picture is displayed on the video monitoring device 18. The display position of the slave picture in this case is one of display positions PL1-PL4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference communication device having a display function of forming a predetermined small area screen in a display full screen and displaying an image different from the display full screen on the small area screen. Regarding control method.

[0002]

2. Description of the Related Art For example, CCITT Recommendation H.264. A video conference communication device has been proposed which has a video conference communication function defined by 320 and which realizes conference communication by exchanging video and audio between remote places. In recent years, in particular, as a communication application using ISDN as a line, a video conference communication device has been regarded as important and is gradually becoming popular.

Further, such a video conference communication device is provided with a display function of forming a predetermined small area screen in the entire display screen and displaying an image different from the entire display screen on the small area screen. Is in practical use. For example, if the received moving image received from the other terminal is displayed on the entire display screen and the transmitted moving image being transmitted from the own terminal to the other terminal is displayed on the small area screen, the user confirms the state of the transmitted moving image. While receiving the received moving image, the video conference communication session can be smoothly performed.

[0004]

However, such a conventional device has the following disadvantages.

That is, the display position of the small area screen displayed on the full display screen is set by the user so as to be opposed to or set to a position peculiar to the apparatus. Therefore, among the images displayed on the full display screen, A small area contains an effective image area that is hidden by the screen (for example, an image of the face part of a person in conversation or the pointing position of the screen when the person points the reference position of the material). If there is, there is an inconvenience that smooth proceedings are impaired.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a control method of a video conference communication device capable of realizing smooth proceedings of proceedings.

[0007]

DISCLOSURE OF THE INVENTION The present invention is a videoconference communication having a display function of forming a predetermined small area screen in a display full screen and displaying an image different from the display full screen on the small area screen. In the control method of the apparatus, the display position of the small area screen is set to a portion avoiding the display area of the effective information of the image displayed on the entire display screen. Also, the valid information of the image displayed on the display full screen is
It may be formed based on the motion vector information of the image.

In addition, a video conference having a display function of displaying a received moving image on the entire display screen, forming a predetermined small area screen in the entire display screen, and displaying the transmitted moving image on the small area screen. In the control method of the communication device, the display position of the small area screen is set to a portion avoiding the display area of the valid information of the received moving image displayed on the entire display screen. The valid information of the received moving image is
The motion vector information obtained when decoding the received moving image information for forming the received moving image may be used.

Further, a video conference having a display function of displaying a transmission moving image on the entire display screen, forming a predetermined small area screen in the entire display screen, and displaying the receiving moving image on the small area screen. In the control method of the communication device, the display position of the small area screen is set to a portion avoiding the display area of the valid information of the transmission moving image displayed on the entire display screen. Further, the valid information of the transmission moving image is
The motion vector information obtained when the transmitted moving image is encoded and compressed can be used.

In addition to the telewriting communication function, a predetermined small area screen is formed in the telewriting display full screen, and a display function for displaying a different image from the telewriting display full screen is provided on the small area screen. In the control method of the video conference communication device, the display position of the small area screen is set to a portion avoiding the effective information display area of the image displayed on the full telewriting display screen. Further, the effective information of the image displayed on the full screen of the telewriting display can be formed based on the telewriting drawing information.

[0011]

Therefore, the small area screen is displayed in a portion of the image displayed on the entire display screen that is not affected even if it is hidden. Therefore, the video conference communication can be appropriately performed while referring to the small area screen. And the proceedings will be smooth.
Also, of the images displayed on the full screen of the telewriting display, the small area screen is displayed in the part that is not affected even when hidden, so the telewriting function is used and
Since the video conference communication can be appropriately performed while referring to the small area screen, the proceeding of the proceedings becomes smooth.

[0012]

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

FIG. 1 shows an example of a video conference communication device according to an embodiment of the present invention. Here, this video conference communication device is CCITT recommendation H.264. The video conference communication function and the telewriting communication function defined by 320 are provided, and ISDN is used as a transmission line.

In FIG. 1, a system control unit 1 is used to control the operation of each unit of the video conference communication device and to execute a video conference communication control procedure process in the information channel (B channel). The (read only memory) 2 is for storing a part of the control processing program executed by the system control unit 1 and various data referred to when executing this program, and is a RAM (random memory). Access memory) 3
The work area of the system control unit 1 is configured.

The clock circuit 4 is for outputting the current time information, and the image processing section 5 is, for example, an image reduction / enlargement process, a still image of a photographic screen, and a still image encoding / decoding process. This is for executing various image processes necessary for the device functions provided by the video conference communication device, such as the digitizing process (JPEG system) and the statistical process of images.

The magnetic disk device 6 includes a system controller 1
The operation display unit 7 is for storing program data such as other control programs and application programs executed by the computer, or other various data.
Is for operating the video conference communication device, and the handwriting input device 8 is for realizing handwriting input and a screen instruction (marker) for the telewriting communication function. The device 9 is for inputting a still image, and comprises, for example, a still image camera device.

The scanner 10 is for reading and inputting an original image at a predetermined resolution, and the printer 11
Is for recording and outputting a recorded image at a predetermined resolution.

The voice input / output device 12 is for exchanging voice information with a partner terminal to realize a conversation between users. The voice CODEC 13 is an analog voice input from the voice input / output device 12. The signal is converted into corresponding digital audio data, the received digital audio data is converted into corresponding analog audio signal, and the analog audio signal is output to the audio input / output device 12. Voice control unit 14
Are for controlling the operation of the voice input / output device 12.

The video camera device 15 is for taking a picture of a participant or the like on the side of the own terminal, and is a video CODEC.
16 is an NTSC output from the video camera device 15.
A video signal of a type is converted to video data of corresponding digital data, the video data is converted to a video data of a predetermined CIF format or QCIF type, and C
CITT Recommendation H.264 In addition to encoding and compressing video information by the encoding method of H.261, the video information in the encoded and compressed state is converted into original CIF or QCIF type video data, and the video data is converted into a corresponding analog signal. Convert to video signal and then convert the video signal to NTSC
It is to be converted into a type video signal.

The video signal converted and formed by the video CODEC 16 is output to the display controller 17. The video signal output from the video camera device 15 is also added to the display controller 17. The display control unit 17 is for controlling the content of the display screen of the video monitor device 18. Further, the video camera control unit 19 is for controlling the camera angle and the like of the video camera device 15.

The ISDN interface control section 20 connects this video conference communication device to ISDN, and
Layer 1 signal processing function, D channel (signal channel)
Signal and two B channel (information channel) signals are integrated / separated.

The D-channel transmission control unit 21 has a call control procedure function performed in a signal channel and a multiplex call control function. The multiplexing / demultiplexing control unit 22 uses the CCI.
TT Recommendation H. 221 is provided with a data multiplexing / separation function conforming to 221 and a data frame synchronization function, and digital audio data,
Digital video data and general-purpose data are added, and the connection end on the multiplexing side is connected to the ISDN interface control unit 20.

These system control unit 1, ROM 2 and R
AM3, clock circuit 4, image processing unit 5, magnetic disk device 6, operation display unit 7, handwriting input device 8, still image input device 9, scanner 10, printer 11, voice CODEC1
3, the audio control unit 14, the video CODEC 16, the display control unit 17, the video camera control unit 19, the D channel transmission control unit 21, and the multiplexing / demultiplexing device 22 are connected to the system bus 23, and each of them is connected. Data is exchanged between the elements mainly via the system bus 23.

Also, the multiplexer / demultiplexer 22 and the voice COD
Audio data is exchanged with the EC 13, and video data is exchanged with the multiplexer / demultiplexer 22 and the video CODEC 16.

Now, CCITT Recommendation H.264. In the video conference communication function defined by 320, the video CODEC1
As a coding method of CCITT Recommendation H.6. The moving picture coding method defined by H.261 is applied. This CCIT
Recommendation H.T. The moving picture coding method of H.261 will be described next. Here, as the moving image information, two types of formats, CIF and QCIF, are defined, but in the following description, only CIF will be described.

CIF (Common Intermed)
The iate Format (Common Intermediate Format) format is a basic format of moving image information, and is a CIF.
Displays a screen of 29.97 frames per second, and for the luminance component (Y), one frame (image frame) is composed of 288 lines, one line is composed of 360 pixels, and the color difference component ( For Cb and Cr), one frame is made up of 144 lines and one line is made up of 180 pixels.

Then, the frame is divided into 12 GOBs (Group Of Blocks) as shown in FIG.
k), and each GOB is divided into 33 macroblocks of 11 × 3 as shown in FIG.

Further, as shown in FIG. 7C, one macroblock has 2 × 2 4 pixels for the luminance component.
The color difference component is divided into 1 × 1 blocks. Therefore, each block of the luminance component and the color difference component is composed of 8 pixels × 8 lines, as shown in FIG.

This moving picture information corresponds to the hierarchical structure of frame / GOB / macroblock / block as described above, an example of which is shown in FIGS. The following description is based on CCITT Recommendation H.264. It is based on TTC standard JT-H261 which is a domestic standard corresponding to 261.

First, the moving picture information of one frame is, as shown in FIG. 9A, a frame start code PSC consisting of 20-bit data of a predetermined bit pattern representing the start of the frame, and a frame number TR representing the sequence number of the frame. 1. Type information PTYPE, which represents information about the entire frame, extension data insertion information PEI, preliminary information PSPARE, and GOB data. Here, the extension data insertion information PEI may include at least one and the preliminary information PSPARE may not be included.
The frame start code PSC, the frame number TR, the type information PTYPE, the extension data insertion information PEI, and the spare information PSPARE are called a frame header.

One GOB data is arranged for each GOB. One GOB data indicates the start of GOB data, as shown in FIG.
OB start code GBSC, GOB number GN indicating the position of GOB, and quantization characteristic information GQUA indicating information on the quantization characteristic.
NT, extension data insertion information GEI indicating presence / absence of extension data area (preliminary information), preliminary information GSPARE, and macroblock data. Here, the extension data insertion information GEI may include at least one and the preliminary information GSPARE may not be included. Also, GOB
Start code GBSC, GOB number GN, quantization characteristic information G
QUANT, extension data insertion information GEI, and spare information GSPARE are called GOB headers.

One macroblock data is arranged for each macroblock. However, if there is no information in that part of the frame, it will not be transmitted.

One macroblock data is, as shown in FIG. 3C, a macroblock address MBA for indicating the position of the macroblock, a type of the macroblock, and a type for indicating which data element appears. Information MTYPE, quantization characteristic information MQUANT representing the quantization characteristic, motion vector information MV
D, a significant block pattern CB representing that it is a macroblock in which at least one transform coefficient is transmitted
It consists of P and block data. Here, the quantization characteristic MQUANT, the motion vector information MVD, and the significant block pattern CBP appear when instructed by the type information MTYPE. Also, the macro block address MBA, the type information MTYPE, the quantization characteristic information M
QUANT, motion vector information MVD, and significant block pattern CBP are called macroblock headers.

As shown in FIG. 3D, the block data includes, for example, a transform coefficient TCOEFF representing transform coefficient data obtained by DCT (discrete cosine transform) processing of image data of the block, and a block coefficient. It is composed of a block end code EOB indicating the end.

In this embodiment, the video monitor device 18 and the video camera device 15 are N
TSC (National TV System Co)
The video C
The ODEC 16 is configured, for example, as shown in FIG.

In the figure, the NTSC / CIF converter 2
5 is a video shooting signal VV from the video camera device 15.
Is converted from the above-mentioned NTSC format signal into a CIF format signal, and its output is added to the encoding processing unit 26 as a signal VC.

The encoding processing unit 26 receives the input signal VC from the TTC standard JT-H. In this case, the coding process of the image coding system defined in H.261 is applied, and the code portion of the output is output to the error correction coding unit 27 as the signal SV, and at the same time, in macroblock units of a predetermined area. The signal representing the value of the motion vector calculated in (1) is output to the system control unit 1 via the system bus 28 as the transmission motion vector signal Smc.

The error correction coding unit 27 buffers the signal SV and, for example, (511,493)
The signal SV is converted into a format in which a predetermined error correction code such as BCH is added, and its output is output to the multiplexer / demultiplexer 22 as a transmission video signal SVa.

Further, the error correction decoding unit 28 performs multiplexing / decoding.
The received video signal RVa input from the separation device 22 is buffered and a predetermined error correction decoding process is applied. The processing result is added to the decoding processing unit 29 as the decoded reception video signal RV. Has been.

The decoding processing section 29 applies the decoding processing of the above-mentioned image coding method to the input decoded reception video signal RV, and the output thereof is the CIF / NTSC conversion section 30 as the signal VR. Has been added to. A signal representing the value of the motion vector calculated by the decoding processing unit 29 in units of macroblocks in a predetermined area is output to the system control unit 1 via the system bus 28 as a reception motion vector signal Rmc. .

The CIF / NTSC converter 30 converts the input CIF format signal VR into an NTSC format signal, and the converted output is added to the display controller 17 as a signal VM.

In this video conference communication device, CCITT Recommendation H.264 is used for the B channel. Data is exchanged in the multi-frame format defined by 221.

As shown in FIG. 5, one multi-frame MFL includes eight sub-multi-frames SMF1 to SMF.
8 sub-multiframes SMF1 to 8
Each SMF 8 is composed of two frames. That is, one multi-frame MFL is composed of 16 frames FLM0 to FLM15.

Each frame FLM0 to FLM15
As shown in FIG. 6, each octet is composed of data of 80 octets, and the bit positions in which these octets are arranged in bit order form subchannels SCH1 to SCH8.

The 8th bit of the 1st octet to the 8th octet is the frame synchronization signal (Frame Ali).
The 8th bit of the 9th octet to the 16th octet constitutes a bit rate allocation signal (Bit rate Allocatio).
n Signal) BAS. Further, in the portion of the 8th bit of the 17th octet to the 80th octet of the sub-channel SCH8, the data of the encryption channel for exchanging key information for encrypting the data is set in a part thereof. Sometimes (optional).

In this way, the frame synchronization signal FAS
8 bits are arranged in one frame FLM0 to FLM15, and the bit allocation is configured in units of multi-frame MFL as shown in FIG.

That is, even frames FLM0, FLM
2, ..., The second to eighth octets of the FLM 14, and the odd frames FLM1, FLM3 ,.
A horizontal synchronization signal composed of an 8-bit data pattern "001101111" is arranged in the FLM 15, and a vertical synchronization composed of a 6-bit data pattern "001011" is arranged in the first octet of the odd-numbered frames FLM1, FLM3, ..., FLM11. A sync signal is placed.

By detecting the horizontal synchronizing signal and the vertical synchronizing signal, the synchronization of one multi-frame MFL can be detected.

The 0th frame, the 2nd frame, the 4th frame
Bits N1, N2, N3, N4, N5 of the first octet of the frame, the sixth frame and the eighth frame are used to indicate the multi-frame number. Of these, the bit N5 is used to indicate whether or not the multiframe number is used. In this way, since the data used for the multi-frame number is 4 bits, the multi-frame number changes in descending order with a value of 0 to 15, and the same multi-frame number appears every 16 multi-frames.

The tenth frame, the twelfth frame,
And bit L of the 1st octet of the 13th frame
1, L2, L3 are used to display the connection number indicating the order in which the information channels carrying the frame among the currently used information channels are connected. The bit R of the first octet of the fifteenth frame is reserved (reserved) for future recommendation and is normally set to 0.

The bit TEA of the first octet of the 14th frame is used to indicate that data transmission is impossible due to an internal failure of the data terminal device.

Also, odd-numbered frames FLM1, FLM3
.., bit A of the third octet of the FLM 15 is used to indicate whether frame synchronization or multi-frame synchronization is established or out of synchronization.

Also, odd-numbered frames FLM1, FLM3
..., bit C1 of the fifth octet, sixth octet, seventh octet, and eighth octet of FLM15
C2, C3 and C4 are for displaying a CRC (Cyclic Redundancy Check) code that is referred to for data error detection (that is, channel quality detection) of two consecutive frames (that is, sub-multiframe). , And the bit E of the fourth octet of the odd frames FLM1, FLM3, ..., FM15 is used to indicate that a transmission error has been detected on the receiving side.

The bit rate allocation signal BAS is, as shown in FIG. 8, even-numbered frames FLM0, FLM2,
.., FLM14, 8-bit data representing the capability BAS or BAS command is arranged, and in the subsequent odd numbered frames FLM1, FLM3 ,.
A double error correction code for error correcting the value of the capability BAS or BAS command transmitted in the immediately preceding frame is arranged.

The multi-frame MFL data is transmitted in the order of frame numbers, and each of the frames FLM0 to FLM15 is transmitted in the octet order from the first octet to the 80th octet, as shown in FIG. The first bit of the octet is sent first.

That is, each frame FLM0 to FLM0
In the FLM 15, the first bit of the first octet is transmitted first and the eighth bit of the 80th octet is transmitted last.

Also, in this video conference communication device, data transmission is normally performed using two information channels, and an example of the transmission procedure at that time is shown in FIG. Note that this transmission procedure is based on CCITT Recommendation H.264. 242 and CCITT Recommendation H.264. Follow 320.

First, the calling terminal calls the destination terminal and performs a call setup procedure on the signaling channel to secure one information channel (hereinafter referred to as the first channel) (Phase A). Frame synchronization is performed while exchanging frame data in which PCM audio data (A-law or μ-law, 64 Kbps) is set on one channel (frame mode). When frame synchronization is established, the capability BAS data and command BAS data are mutually exchanged. Exchanges (phase B1-1), decides a transmission mode to be used at that time, and initiates an additional call setup request for securing the second information channel (phase B1-2).

Then, from the contents of the exchange of the transmission modes at that time, the common and highest function mode is selected (phase B1-3), and the calling terminal functions as the called terminal in the selected operation mode. A BAS command for designating a terminal function is operated to operate, and parameters common to the device functions of the calling terminal and the called terminal are set (phase B2). Thereby, in the first channel, for example, data transmission of audio data (16 Kbps) and data transmission of moving image data (46.4 Kbps) corresponding to the transmission mode selected at that time are performed (phase C). .

When the first channel starts data transmission in the frame mode, the call setup procedure is performed on the signaling channel for the second information channel (hereinafter referred to as the second channel) (phase CA), and the second channel is established. Then, frame data including only the frame synchronization signal FAS and the bit allocation signal BAS is exchanged using the second channel to establish frame synchronization and multi-frame synchronization (phase CB1-11), and then the first channel and the second channel. Establish synchronization between channels (Phase CB1-1
2).

When the synchronization of the two information channels is completed,
The BAS command is sent from the calling terminal side to set the transmission mode (phase CB1-2), the transmission mode is switched to the set contents (phase CB1-3), and the common parameter is set (phase CB3).

In this way, when the initialization of the second channel is completed, after that, the frame data exchanged in the first channel and the frame data exchanged in the second channel are synchronized, for example, Audio data, moving image data, and general-purpose data are 56 Kbps and 62.4 K, respectively, using two information channels.
Data transmission is performed by allocating the transmission rate of bps and 6.4 Kbps (see FIG. 11).

To end such data transmission, first, the second channel is disconnected. At this time, the first
For voice data transmission on the channel only, the procedure for setting the common mode is performed (Phase CD
1), the mode of the second channel is switched to the mode 0F of the frame mode (phase CD2). At this time, the first channel and the second channel are asynchronous, and the second channel is in a state where the call is held in the transmission state of only the frame synchronization signal FAS and the bit allocation signal BAS, and the call disconnection of the signal channel is performed. The call for the second channel can be released by the release procedure.

In the first channel, phase CD1
During phase CD2, in the frame mode, for example, voice data and moving image data are transmitted at a total transmission rate of 62.4 Kbps, and when the operator of one terminal ends the call, the transmission of the moving image data is ended. However, in order to use the transmission capacity of all the first channels including the transmission capacity of this moving image data for audio transmission, mode 0
After switching to F (Phase D2), the call of the first channel can be released by the call disconnection release procedure of the signaling channel.

As a result, the call disconnection release procedure is executed on the signaling channel for the first channel and the second channel (phase E), and the video conference communication between the two terminals is completed.

As described above, in the video conference communication device, first, one information channel (first channel) is secured and the frame mode is established, and then the transmission rates of the audio data and the moving image data are assigned to the first channel. If the second information channel (second channel) can be set at the same time while performing the data transmission in the transient mode, the second channel is secured by the call setting procedure of the signal channel. And the first transmitting in the transient mode
In order to utilize the increased transmission capacity of the information channel by synchronizing the channel with the newly secured second channel and resetting the allocation of the transmission rate of audio data and video data when the channel synchronization is established. In addition, the encoding rules (encoding method) of audio data and moving image data are changed so that higher quality audio data and moving image data can be exchanged.

Further, when the data transmission is completed, it is necessary to temporarily change the mode in which the transmission path in which both the first channel and the second channel are synchronized is used to the mode in which only the first channel is used. is there. Therefore, in this case, first, the encoding rules of the audio data and the moving image data are set to a method that optimizes the transmission capacity of the first channel of 62.4 Kbps, and the transmission mode of only the first channel is changed. , In the second channel, the state of synchronization with the first channel is stopped, and after transitioning to mode 0F in which user data is in an idle transmission state, the call disconnection release procedure of the signal channel is executed to disconnect the call. /release. Also, the first channel disconnects / releases the call by the call disconnection release procedure of the signal channel after changing from two types of media transmission of audio data and moving image data to mode 0F which is a transmission mode of only audio data. . The first channel can also be disconnected / released after the mode is directly changed to the mode 0F without switching the mode to the two types of media transmission of the audio data and the moving image data. Also,
The billing information obtained in the call disconnection / release procedure is managed for each channel.

Now, in this embodiment, as shown in FIG. 12, an in-screen screen display function for displaying a small screen on the display screen of the video monitor device 18 is provided. Here, a screen that occupies the entire display screen is referred to as a parent screen PP,
Further, a small screen displayed on the parent screen PP is referred to as a child screen PC.

This child screen PC is, for example, as shown in FIG. 13, a display position PL set at four positions on the parent screen PP.
It is displayed on any one of 1, PL2, PL3 and PL4. The display position of the child screen PC is basically designated by the user, but in the present embodiment, the child screen PC is automatically placed at a position where the effective portion of the image displayed on the parent screen PP is not hidden. It is equipped with an operating mode that moves automatically.

The image composition of the parent screen PP and the child screen PC will now be described. This image synthesizing process is executed by the display control unit 17.

The display control unit 17 internally forms a virtual screen, and stores a virtual image memory corresponding to the virtual screen in FIG.
Set as shown in (a). This virtual image memory is set in a predetermined area of the RAM 3.

In the virtual image memory, the video monitor device 1
A region VP for storing the display data displayed on the parent screen PP of 8 and regions VCa, VCb forming a double buffer for storing the display data displayed on the child screen PC are formed. In this case, the display data stored in the areas VCa and VCb is either display data in a state in which an image is cut out from the original screen at the same size (clipping) or display data in a state in which the original screen is reduced. Used.
Further, the selection of the type of the display data stored in the areas VCa and VCb is appropriately performed according to the user setting.

Here, the display data of the child screen PC is alternately written in the area VCa and the area VCb in frame units, and is read from the area VCa, VCb in which the display data is not written at that time. The displayed display data is combined with the display data read from the area VP.

In this synthesizing method, for example, the display data writing position AP to the area VP and the display data reading position BP from the area VP have a relationship as shown in FIG. When reading the data of the small screen PC from VCa, the read position BP from the area VP
However, when it is in a state of being included in the area corresponding to the composite position of the child screen PC, as shown in FIG. 7C, the read position BP is moved to the corresponding position of the area VCa,
Display data is read from the area VCa.

As a result, the screen in which the child screen PC is superimposed on the parent screen PP (see FIG. 12) is displayed on the video monitor device 1.
8 is displayed.

Now, when the child screen PC is combined with the effective area of the image displayed on the parent screen PP, the intention of the speaker is lost and the instructed material is hidden. Hinders the top.

Therefore, in this embodiment, it is determined that the image portion (moving image portion) in which the display content of the main screen PP is changed is the effective area of the image at that time, and it is deviated from the effective area as much as possible. The small screen PC is displayed on the part. The display position of the child screen PC in this case is as shown in FIG.
This is one of the four display positions PL1, PL2, PL3, PL4 shown in FIG.

The judgment of the effective area is made by, for example, the parent screen P.
When P is a received image, the received motion vector signal Rmc output from the video CODEC 16 is used. This reception motion vector signal Rmc is 1 as shown in FIG.
One can be obtained for each area unit (corresponding to a 4 × 4 macroblock unit) obtained by dividing the screen into 11 × 9 screen areas.
A plurality of reception vector signals Rm obtained from areas corresponding to the respective display positions PL1, PL2, PL3, PL4
The sum of the absolute values (scalar) of c is calculated, and the sum is multiplied by a predetermined coefficient to display positions PL1, PL2, P.
The evaluation values of L3 and PL4 are calculated, and the display positions PL1, PL2, PL3 and PL4 having the smallest evaluation values are determined as the display positions of the child screen PC.

By the way, when the telewriting communication function is used and the child screen PC is combined with the parent screen PP, the screen as shown in FIG. 16 is displayed on the display screen of the video monitor device 18. To be done. In this case, a telewriting screen is displayed on the parent screen PP, and a received moving image is displayed on the child screen PC.

In this case, the display control unit 17 displays the information shown in FIG.
As shown in, the virtual image memory corresponding to the virtual screen is set.

That is, in the virtual image memory, an area VP for storing the display data displayed on the parent screen PP of the video monitor device 18 and a double buffer for storing the display data displayed on the child screen PC are formed. Area VC
In addition to a and VCb, a region VT for storing drawing data of the telewriting communication function and display data of the marker (screen indicator) is formed.

In the image synthesizing method in this case, for example, the display data writing position AP to the region VP, the display data reading position BP from the region VP, and the reading position CP from the region VT are shown in FIG. When the relationship is as shown in a), the read position B from the area VP
When P coincides with the coordinates of the drawing data, the data is read from the area VT as shown in FIG.
Further, when the read position BP is in a state of being included in the area corresponding to the composite position of the small screen PC, the read position BP is moved to the position corresponding to the area VCa to display data from the area VCa in the same manner as described above. Read out.

As a result, a screen in which the child screen PC is superimposed on the parent screen PP of the telewriting screen (see FIG. 16)
Is displayed on the video monitor device 18.

In this case, if the child screen PC hides the drawing position of the telewriting screen, the telewriting communication will be hindered. Therefore, in this embodiment, the display positions PL1, PL2, PL3, PL4 including the drawing data are included.
By displaying the child screen PC in the part excluding, the situation where the child screen PC hides the drawing position of the telewriting screen is avoided.

FIG. 19 shows an example of processing executed by the system control unit 1 to set the display positions PL1, PL2, PL3, PL4 for displaying the small screen PC. It should be noted that this process is executed for each frame of the moving image display.

First, it is checked whether or not the telewriting communication function is being used at that time (decision 101). When the result of the judgment 101 is YES, it is determined whether or not the telewriting screen is set as the main screen PP. Check (decision 102).

When the result of the determination 102 is YES, the display positions PL1, PL2 of the child screen PCs are displayed.
For PL3 and PL4, a telewriting drawing evaluation value calculation process (process 103) for calculating an evaluation value representing the ratio of the drawing data exchanged with the partner terminal at that time is executed.

Further, when the result of the judgment 101 is NO and when the result of the judgment 102 is NO,
This is a case where the transmission moving image or the reception moving image is displayed on the main screen PP. In this case, it is checked whether or not the image displayed on the main screen PP is a transmission moving image (decision 10
4) When the result of determination 104 is YES, the sum of absolute values (scalar) of the plurality of transmission vector signals Smc obtained from the areas corresponding to the respective display positions PL1, PL2, PL3, PL4 is calculated, Multiplying the sum by a predetermined coefficient, each display position PL1, PL2, PL
3, a coding side motion evaluation value calculation process (process 105) for calculating the evaluation value of PL4 is executed.

When the result of the judgment 104 is NO, the display positions PL1, PL2, PL3, P respectively.
The sum of absolute values (scalars) of the plurality of reception vector signals Rmc obtained from the area corresponding to L4 is calculated, and the sum is multiplied by a predetermined coefficient to display positions PL1, PL.
Decoding side motion evaluation value calculation processing (processing 106) for calculating evaluation values of 2, PL3 and PL4 is executed.

When the evaluation value for one frame is calculated in this way, the display position PL1, which has the smallest evaluation value,
A combined area determination process (process 107) for determining PL2, PL3, and PL4 as display positions of the child screen PC is executed. As a result, when the display position of the child screen PC is determined,
The determination result is output to the display control unit 17 (process 10
8) Specify the display position of the small screen PC.

FIG. 20 shows an example of the telewriting drawing evaluation value calculation processing.

First, it is checked whether or not drawing is being performed at that time (decision 201). If the result of the judgment 201 is NO, this process is immediately terminated. When the result of determination 201 is YES, one display position (area) for displaying the child screen PC is selected (process 202), and the coordinate area of the display position includes a series of drawing data at that time. It is checked whether or not it is determined (decision 203).

When the result of judgment 203 is YES,
An in-drawing mark for storing that the image is being drawn is set in the area corresponding to the display position (process 204), and the evaluation value is calculated by multiplying the data amount of the drawing data included in the area by a predetermined coefficient. (Processing 205), the calculated evaluation value is stored in association with the display position (Processing 20).
6).

When the result of judgment 203 is NO, it is checked whether or not the drawing mark corresponding to the display position selected at that time is set (decision 20).
7). When the result of the determination 207 is YES, it means that the drawing position is right outside the display position. In this case, the drawing mark is reset (process 208).

Further, when the result of the judgment 207 is NO, it means that there is no drawing at the display position selected at that time, so the evaluation value stored corresponding to the display position is "0". (Process 209).

In this way, when the processing for one display position is completed, it is checked whether or not the processing for all display positions is completed (decision 210). If the result of the judgment 210 is NO, the process 202 Then, the process for the next display position is executed. Also, the decision 210
When the result of is YES, this process ends.

FIG. 21 shows an example of the coding side motion evaluation value calculation process.

First, the encoding processing unit 26 of the video CODEC 16 selects one processing unit block (corresponding to a 4 × 4 macro block) for calculating a motion vector (processing 301), and the processing unit block is encoded. Wait until the end (NO loop of decision 302), decision 302
If the result is YES, the value of the transmission motion vector signal Smc at that time is input (process 303), and the input value is stored in association with the processing unit block at that time (process 304).

At that time, it is checked whether or not the processing for all processing unit blocks has been completed (determination 30
5) If the result of determination 305 is NO, process 3
Returning to 01, the processing of the next processing unit block is executed.

When the processing is completed for all processing unit blocks and the result of determination 305 is YES, the display positions PL1, PL2, PL3, PL4 are saved for the plurality of processing unit blocks included. The sum of absolute values of the transmission motion vector signal Smc is calculated, and the sum is multiplied by a predetermined coefficient to calculate an evaluation value,
The evaluation value is saved (process 306).

FIG. 22 shows an example of the decoding side motion evaluation value calculation process.

First, the decoding processing unit 29 of the video CODEC 16 selects one processing unit block (corresponding to a 4 × 4 macro block) for calculating a motion vector (processing 401), and decoding is performed on the processing unit block. Wait until the end (NO loop of decision 402), decision 402
If the result is YES, the value of the received motion vector signal Rmc at that time is input (process 403), and the input value is stored in association with the processing unit block at that time (process 404).

At that time, it is checked whether or not the processing has been completed for all processing unit blocks (decision 40
5), when the result of determination 405 is NO, process 4
Returning to 01, the processing of the next processing unit block is executed.

When all the processing unit blocks have been processed and the result of the determination 405 is YES, the display positions PL1, PL2, PL3, PL4 have been saved for the plurality of processing unit blocks included. The sum of absolute values of the reception motion vector signal Rmc is calculated, and the sum is multiplied by a predetermined coefficient to calculate an evaluation value.
The evaluation value is saved (process 406).

FIG. 23 shows an example of the combined area determination process.

First, each display position PL1, PL2, PL
3, the evaluation value stored for PL4 is read (process 501), and the evaluation value is displayed at the minimum display position PL1,
PL2, PL3 and PL4 are determined (process 502), and the determination result is output (process 503).

As described above, in the present embodiment, when the in-screen screen function is applied, the child screen is displayed in a portion as far as possible from the effective area (moving image portion) of the display contents of the parent screen. Therefore, it is possible to avoid the fact that the image of the important screen is hidden, and thus it is possible to smoothly operate the video conference communication.

Further, when the telewriting communication function is used, the sub-picture is displayed in the portion excluding the position referred to in the telewriting communication, so that writing or reference is performed in the telewriting communication. The part is not hidden, and therefore, the video conference communication using the telewriting communication function can be executed more smoothly.

By the way, in the above-described embodiment, the motion detecting function (motion compensation function) of the video CODEC is used to detect the moving image portion of the main screen. However, even when other means is used, The moving image part can be detected. For example, a block matching detection means for that purpose may be provided, or a means for detecting a moving image portion using a well-known gradient method may be used.

Further, in the above-mentioned embodiment, the moving part of the image is determined as the effective part of the image. However, for example, the moving part of the person's face or hand, or
It is also possible to detect a significant image portion such as a character of a document by an image recognition method and determine the detected portion as an effective portion of the image.

[0111]

As described above, according to the present invention,
Of the images displayed on the full screen, the small area screen is displayed in the part that is not affected even if it is hidden, so it is possible to properly perform video conference communication while referring to the small area screen. Progress will be smooth. In addition, since the small area screen is displayed in the part of the image displayed on the full screen of the telewriting display that is not affected even when it is hidden, the telewriting function can be used while referring to the small area screen. Since the video conference communication can be appropriately performed, the proceeding of the proceedings can be smoothly performed.

[Brief description of drawings]

FIG. 1 is a block diagram showing a video conference communication device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing an example of a signal format of moving image information.

FIG. 3 is a schematic diagram showing an example of a data format of moving image information.

FIG. 4 is a block diagram showing an example of a video CODEC.

FIG. 5 is a schematic diagram showing an example of a frame configuration of multi-frame.

FIG. 6 is a schematic diagram showing an example of a signal configuration of one frame.

FIG. 7 is a schematic diagram showing an example of a frame adjustment signal.

FIG. 8 is a schematic diagram showing an example of a bit rate allocation signal.

FIG. 9 is a schematic diagram for explaining a signal transmission order.

FIG. 10 CCITT Recommendation H. The time chart which showed the example of the general procedure of 242.

FIG. 11 shows audio data when two data channels are secured and frame mode data transfer is executed.
FIG. 3 is a schematic diagram showing an example of allocation of transmission rates of general-purpose data and moving image data.

FIG. 12 is a schematic diagram showing a display example of an in-screen screen.

FIG. 13 is a schematic diagram showing an example of a display position of a child screen.

FIG. 14 is a schematic diagram showing an example of screen synthesis.

FIG. 15 is a schematic diagram for explaining a motion vector calculation unit.

FIG. 16 is a schematic diagram showing a display example of an in-screen screen when using a telewriting communication function.

FIG. 17 is a schematic diagram showing an example of a virtual screen in the case of FIG.

FIG. 18 is a schematic diagram for explaining screen composition in the case of FIG. 16;

FIG. 19 is a flowchart showing an example of processing executed by the system control unit to set a display position for displaying a child screen.

FIG. 20 is a flowchart showing an example of a telewriting drawing evaluation value calculation process.

FIG. 21 is a flowchart showing an example of a coding side motion evaluation value calculation process.

FIG. 22 is a flowchart showing an example of decoding-side motion evaluation value calculation processing.

FIG. 23 is a flowchart showing an example of a combined area determination process.

Claims (8)

[Claims] 1. A control method for a video conference communication device, comprising: a display function of forming a predetermined small area screen in the entire display screen and displaying an image different from the entire display screen on the small area screen. A method for controlling a video conference communication device, wherein the display position of the screen is set in a portion avoiding the display area of the valid information of the image displayed on the entire display screen. 2. The method of controlling a video conference communication device according to claim 1, wherein the valid information of the image displayed on the full-screen display is formed based on motion vector information of the image. 3. A reception moving image is displayed on the entire display screen, and a predetermined small area screen is formed in the entire display screen,
In the control method of the video conference communication device having the display function of displaying the transmission moving image on the small area screen, the display position of the small area screen is changed to the display area of the valid information of the reception moving image displayed on the entire display screen. A method for controlling a video conferencing communication device, characterized in that it is set in a portion avoiding. 4. The television according to claim 3, wherein the valid information of the received moving image uses motion vector information obtained when decoding the received moving image information for forming the received moving image. A method for controlling a conference communication device. 5. A transmission moving image is displayed on the entire display screen, and a predetermined small area screen is formed in the entire display screen.
In the control method of the video conference communication device having the display function of displaying the received moving image on the small area screen, the display position of the small area screen is changed to the display area of the valid information of the transmitted moving image displayed on the entire display screen. A method for controlling a video conferencing communication device, characterized in that it is set in a portion avoiding. 6. The method of controlling a video conference communication device according to claim 3, wherein the valid information of the transmission moving image uses motion vector information obtained when the transmission moving image is encoded and compressed. 7. A telewriting communication function is provided, and a predetermined small area screen is formed in the telewriting display full screen, and a display function for displaying a different image from the telewriting display full screen on the small area screen is provided. In the video conference communication device control method described above, the display position of the small area screen is set to a portion avoiding the display area of the effective information of the image displayed on the full telewriting display screen. Device control method. 8. The control method of the video conference communication device according to claim 7, wherein the effective information of the image displayed on the full screen of the telewriting display is formed based on the telewriting drawing information.