JP2010147939A - Video communication apparatus, video communication method and video communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic conference system capable of easily specifying a cause of interruption when a sub input video image is interrupted. <P>SOLUTION: The present invention relates to a video communication apparatus 100 comprising: a main video input section 101; a sub video input section 102, a transmission data generating section 105 for generating transmission data including a main video image and a sub video image; a communication section 106 for transmitting transmission data and receiving reception data; a display data generating section 107 for generating display data from reception data; a video output section 108 for outputting display data onto a display 21; a signal detecting section 103 for detecting whether to acquire a sub video image; and a notification section 104 for notifying the transmission data generating section 105 and the display data generating section 107 of interruption information in the case where a sub video signal is not acquired. When the interruption information is reported, the transmission data generating section 105 generates transmission data including the interruption information and the display data generating section 107 generates display data including the interruption information. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a video communication apparatus that transmits and receives a video signal via a communication network, and more particularly to a video communication apparatus having a content sharing function.

In recent years, with the spread of broadband, the use of electronic conferencing systems has become popular as a method for communication between participants in remote locations using video communication lines in real time to exchange video and audio. Furthermore, the electronic conference system not only includes the video of the conference as the main input video, but also has a content sharing function for sharing sub-input video such as PC input for sharing conference materials and sub-camera input for projecting the hand. There are also things. In a system having a content sharing function, the conference can be more smoothly proceeded by switching between the main input video and the sub input video (see Patent Document 1).
JP 2008-46364 A

  However, the video communication apparatus having the conventional content sharing function has a problem that when the sub input video is blocked, it takes a lot of time to identify the cause of the blockage. Specifically, it is as follows.

  In an electronic conference system having a content sharing function, a user can switch inputs by providing a sub-input video interface. At this time, when the sub input video is blocked due to unintentional disconnection of the cable or the power of the sub camera being cut off, a no-signal state of screen display such as a black screen occurs.

  In this case, it takes time to identify the cause of whether the communication with the counterpart device is interrupted due to a failure of the communication network, or whether the communication path is secured but only the sub input is blocked.

  On the other hand, in the electronic conference system described in Patent Document 1, means for switching video by a user operation is disclosed, but blocking of input unintended by the user is not disclosed. For this reason, it takes time to specify the cause of the no-signal state of the screen display, which hinders the progress of the conference.

  Accordingly, the present invention has been made in view of the above-described present situation, and provides a video communication apparatus, a video communication method, and a video communication system that can easily identify the cause of a block when a sub-input video is blocked. The purpose is to do.

  In order to solve the above-described conventional problem, a video communication apparatus that communicates with another video communication apparatus, the first input means for acquiring a first video signal indicating the first video, and the second video Second input means for obtaining a second video signal, transmission data generation means for generating first transmission data including at least the second video among the first video and the second video, and the transmission data generation Communication means for transmitting the first transmission data generated by the means to the other video communication apparatus and receiving the reception data including the third video from the other video communication apparatus, and received by the communication means Display data generating means for generating display data including the third video using received data, output means for outputting the display data generated by the display data generating means to the first display device, and the second input Signal detecting means for detecting whether or not the second video signal is acquired by the means, and the second video signal when the signal detecting means detects that the second video signal is not acquired. And a notification means for notifying the transmission data generation means and the display data generation means of the generated cutoff information, wherein the transmission data generation means When notified, the second transmission data including the blocking information is generated, and the communication unit transmits the second transmission data generated by the transmission data generation unit to the other video communication device, and the display data When the blocking information is notified, the generation unit generates the display data including the blocking information and the third video.

  As a result, both the counterpart device (the side receiving the sub input signal) and the own device (the side transmitting the sub input signal) receive blocking information indicating that the second video signal (for example, the sub input signal) has not been acquired. For example, the user using the counterpart device can recognize that there is a problem in the input of the sub input signal, not the abnormality in communication between the devices and the malfunction of the display device. .

  Also, a user who uses the device itself may not be aware that the sub input signal is not acquired, for example, when the sub input signal is not displayed on the display device on the device side. On the other hand, in the video communication apparatus of the present invention, since the blocking information is also notified to the own apparatus, the user using the own apparatus can search for and recover the cause that the sub input signal is not acquired.

  The transmission data generating unit generates the first transmission data so that the second video is displayed on the second display device by the other video communication device, and the notification unit further switches the display. A display switching request indicating that the first video is displayed on the second display device when the transmission data generating unit is notified of the display switching request. The second transmission data may be generated.

  Thereby, while the sub input signal is acquired, the second video (for example, sub video) is displayed on the display device on the other side, and when the sub input signal is not acquired, the first video (for example, , The main image) is automatically displayed, so that it is not necessary to place a burden on the user who uses the counterpart device to switch the image. Therefore, even when the user is not familiar with machine operations, an electronic conference using this apparatus can be executed smoothly.

  Further, the transmission data generating means displays the second video in a predetermined first area of the second display device, and displays the first video in a second area having a size equal to or smaller than the first area. The first transmission data may be generated as described above, and when the display switching request is notified, the second transmission data may be generated so that the first video is displayed in the first area.

  As a result, while the sub input signal is acquired, the second video (for example, sub video) is displayed on the large screen of the counterpart display device, and the first video (for example, main video) is displayed on the small screen. When the sub input signal is no longer acquired, the display is automatically switched so that the main image is displayed on the large screen. It will be over. Therefore, even when the user is not familiar with machine operations, an electronic conference using this apparatus can be executed smoothly.

  The notification unit may generate the blocking information when it is detected that the second video signal is not acquired continuously for a first period predetermined by the signal detection unit.

  As a result, a predetermined period (first period) is ensured from when the sub input signal is interrupted until notification of the interrupt information, so that, for example, when the sub input signal is instantaneously interrupted and immediately recovered. It is possible to prevent the blocking information from being notified.

  In addition, the notification unit may notify the transmission data generation unit of the display switching request after a predetermined second period has elapsed since the generation of the blocking information.

  Thus, a predetermined period (second period) is ensured until the display is switched after notifying the user that the sub input signal has been cut off, so that the user can restore the sub input signal during this period. . Therefore, flickering of the screen due to frequent screen switching can be prevented.

  Further, the notifying unit further notifies the transmission data generating unit of a release request indicating that only the first video is displayed after a predetermined third period has elapsed since the display switching request was notified. The transmission data generation means may generate the second transmission data so that the first video is displayed on the full screen when the release request is notified.

  As a result, when a sufficient period of time has elapsed since the sub input signal was cut off, it is considered that content sharing has ended, and only the main video can be displayed.

  The transmission data generation means may detect that the second video signal is reacquired after the signal detection means detects that the second video signal has not been acquired. The first transmission data including the second video may be generated.

  As a result, when the sub input signal is restored, the display can be automatically restored to the original, so even if the user is not familiar with machine operations, etc., electronic conferences using this device can be executed smoothly. can do.

  The notification unit may output the display switching request to the transmission data generation unit simultaneously with receiving a stop command indicating that the second video signal is to be stopped from the user.

  As a result, when the user intentionally stops, the display is automatically switched immediately, so that even if the user is not familiar with the machine operation or the like, the electronic conference using the apparatus is smoothly executed. be able to.

  The transmission data generation means may generate the second transmission data so that the blocking information is displayed as character information in a predetermined third area of the second display device.

  The transmission data generation means may further generate the second transmission data so that the blocking information is output as a sound.

  The video communication apparatus may further include an information display unit that displays the blocking information in the display data.

  Note that the present invention can be realized not only as a video communication apparatus but also as a method in which processing means constituting the video communication apparatus is used as a step. Moreover, you may implement | achieve as a program which makes a computer perform these steps. Furthermore, it may be realized as a recording medium such as a computer-readable CD-ROM (Compact Disc-Read Only Memory) in which the program is recorded, and information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

  In addition, some or all of the constituent elements included in each of the video communication apparatuses may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically includes a microprocessor, a ROM, a RAM (Random Access Memory), and the like. Computer system.

  According to the video communication device of the present invention, when the sub input video is blocked, the cause of the blockage can be easily specified.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a configuration of a communication system that realizes a communication service at two locations in the present embodiment.

  In FIG. 1, a communication device A <b> 11 located at the site A and a communication device B <b> 12 located at the site B are video / audio control devices having a communication function and can be interconnected via the network 10.

  The communication device A11 inputs real-time video / audio data at the site A from a camera and a microphone included in or connected to the own device, and transmits the data to the communication device B12 via the network 10. Further, the communication device A11 receives real-time video / audio data at the base B from the communication device B12, and outputs it to a display and a speaker included in or connected to the own device.

  In addition, in order to provide a more realistic communication service that reduces the influence of distance despite the network 10, the communication device A11 and the communication device B12 each include a plurality of displays, cameras, microphones, I / O devices such as speakers are provided. These input / output devices are arranged in advance at suitable positions, and are characterized by this arrangement. This will be described in detail later with reference to the drawings.

  The network 10 may be a wired line or a wireless line, or may be a combination of both. Moreover, it may be a public network such as the Internet or a public telephone line, or may be a local network closed in a limited domain such as a LAN (Local Area Network), or both of them. It may be a combination.

  For example, digital communication using RTP (Real-time Transport Protocol), RTCP (RTP Control Protocol), or TCP (Transmission Control Protocol) is used for communication between the communication device A11 and the communication device B12.

  In addition, it is assumed that each of the communication device A11 and the communication device B12 is assigned an IP address as address information indicating a position on the network. Other information such as a telephone number may be used as the address information instead of the IP address.

  The data transmitted and received by the communication device A11 and the communication device B12 is real-time video / audio data. However, the video / audio data recorded on a storage medium such as an optical disk or a hard disk is also transmitted / received together with the real-time video / audio data. be able to. Note that only video data or audio data may be transmitted and received. Still image data or text data or document data such as HTML (HyperText Markup Language) may be used.

  As described above, the communication device A11 and the communication device B12 can output real-time video and audio of other bases using a plurality of input / output devices arranged in advance at appropriate positions. Communication services can be provided.

  Note that not only the communication service at the two bases shown in FIG. 1 but also the communication service by the interconnection at three or more bases is possible.

  FIG. 2 is a diagram illustrating an example of a configuration of a communication system including devices other than the communication device A11 and the communication device B12 described above.

  In FIG. 2, a communication device A11, a communication device B12, a notebook PC (personal computer) 13, a PDA (Personal Digital Assistant) 15, a mobile phone 16, and a desktop PC 19 are connected, and a communication system at five locations. To implement. Here, it is assumed that the network 10 as a public telephone line and the Internet 18 are connected via a server 17 of an Internet service provider. The desktop PC 19 is connected to the Internet 18, and is connected to other devices such as the communication device A <b> 11 via the Internet 18, the server 17, and the network 10.

  Since the communication device A11 and the communication device B12 are the same as those in FIG. 1, other devices will be described. Note that, as shown in FIG. 2, some or all of the components constituting the communication device A11 and the communication device B12 may be configured by one system LSI.

  The notebook PC 13 does not have a built-in camera function, and the camera 14 is connected externally. The camera 14 is a device capable of moving image shooting such as a digital video camera. The notebook PC 13 transmits the video / audio data captured by the camera 14 to other devices via the network 10 as real-time video / audio data at the local site. When the camera 14 does not have a moving image shooting function but has only a still image shooting function, the shot still image data may be transmitted at regular intervals.

  The PDA 15 does not have a camera function and cannot transmit real-time video data of its own base. The PDA 15 outputs the received video / audio data to the display and the speaker via the network 10 and transmits real-time audio data at the local site to other devices. If the camera function is provided, video / audio data can be transmitted and received.

  The mobile phone 16 is a mobile phone with a camera such as a CCD (Charge Coupled Device) camera, and inputs real-time video and audio data at the local site from the camera and microphone and transmits them to other devices via the network 10. Also, the received video / audio data is output to the display and speaker of the device itself. In addition, the mobile phone 16 includes a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a GSM (Global System for Mobile Communications) system, and a W-CDMA (Wide CDMA (Wide CDMA) system. Any communication method such as Division Multiple Access 1x) or LTE (Long Term Evolution) may be used. In addition, it has a slot that can be loaded with storage media such as SD cards, so that data recorded on the recording media can be shared with other devices participating in the communication service. is there. Further, the mobile phone 16 may connect to the network 10 using other wireless communication functions such as WiMAX (Worldwide Interoperability for Microwave Access).

  The desktop PC 19 has a built-in camera function, inputs real-time video and audio data at its own site from the camera and microphone, and transmits them to other devices via the Internet 18 and the network 10. It is assumed that the Internet 18 and the network 10 are connected via a server 17 of an Internet service provider. Also, the received video / audio data is output to the display and speaker of the device itself. The desktop PC 19 has a device capable of reading a storage medium such as an optical disk or an SD card, an external HDD (Hard Disk Drive), or an internal memory, and is recorded on these devices. Data can be shared with other devices participating in the communication service.

  Each device is assigned a telephone number or an IP address as address information indicating a position on the network. Note that by using an IP address compatible with IPv6 (Internet Protocol Version 6), even if each device physically moves, it is possible to participate in the communication service using the same address.

  In addition, each device may transmit video / audio data by multicast to other devices participating in the communication service. In addition, a specific device (for example, the communication device A11) may be set as a server, and after the server receives and processes video / audio data from the other device, it may be multicast to the other device.

  As described above, the communication device A11 and the communication device B12 can output real-time video and audio transmitted from each device located at a plurality of locations, and can provide more realistic communication services.

  Next, the arrangement of input / output devices included in communication device A11 (or communication device B12) in the present embodiment will be described. FIG. 3 is a diagram illustrating an example of an arrangement of input / output devices included in the communication device according to the present embodiment.

  The communication device A11 includes a main body 20, a display 21, a main camera 22, a microphone 23, speakers 24a, 24b and 24c, a remote controller 26, a sub camera 27, and a notebook PC 28. Each input / output device (display 21, main camera 22, microphone 23, speakers 24a, 24b and 24c, remote control 26, sub camera 27, and notebook PC 28) is connected to the main body 20. This connection may be a wired line or a wireless line. Further, it is assumed that one or more users participating in the communication service are seated at the desk 25 facing the display 21.

  The main body 20 is an information processing apparatus including a CPU (Central Processing Unit) and a memory. The main body 20 controls each input / output device, encoding processing of video / audio data input from the input / output device, communication control processing via the network 10, decoding processing of video / audio data received via the network 10, and The output processing of the decoded video / audio data to the input / output device is performed.

  The display 21 is a device that displays an image and the like, and is, for example, an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel). It is arranged side by side on the front surface of the desk 25 so as to be positioned in front of the user participating in the communication service. Here, one display is connected to the main body 20. Here, although there is one display, a plurality of displays may be used. In this case, the images of the participants at other bases are displayed as if they are seated on the desk 25. The video of another base may be divided and displayed according to the number of displays.

  The main camera 22 is a photographing device having a moving image photographing function such as a digital video camera. A user who is placed at the top of the display 21 and takes a seat on the desk 25 is photographed.

  The microphone 23 is an input device that collects surrounding sounds. Arranged at the center of the desk 25. Moreover, you may arrange | position the directional microphone of the number according to the number of the users who sat on the desk 25 so that it may be located in front of each user. One omnidirectional microphone and one or more directional microphones may be arranged in combination. As a result, it is possible to grasp the direction from which the sound is heard, and it is possible to control the direction in which the sound is output at another site.

  The speakers 24 a, 24 b and 24 c are arranged behind the display 21. Here, three speakers are connected. Accordingly, it is possible to control the speaker that outputs sound in accordance with the video displayed on the display 21.

  The remote control 26 is an operation input device that receives an input instruction from a user and transmits an operation input signal to the main body 20. It may be operable by a user who sits at the desk 25. Although the remote controller is used here, other operation input devices such as a one-touch button, a touch panel, a keyboard, or a mouse attached to the main body 20 may be used. Moreover, you may arrange | position the operation input device of the number according to the number of the users who sat on the desk 25 so that it may be located in front of each user.

  The sub camera 27 is an imaging device for zooming and shooting the hand of the conference person, and is used for sub input video. The main body 20 is connected using a D terminal cable, an HDMI (High-Definition Multimedia Interface) cable, or the like. The sub camera 27 is not fixed, and the conference person may take a picture with his hand. Further, it may be a microscope-type fixed camera.

  The notebook PC 28 is a PC for sharing meeting materials. The notebook PC 28 is connected to the main body 20 with a VGA (Video Graphics Array) cable or the like. The notebook PC 28 has an external display display mode, and transmits an input signal to the main body 20 as a sub input when the external display display mode is ON.

  As described above, the communication device A11 can provide a communication service with another base. That is, the user operates the remote control 26 to set a connection destination (another base), establish communication, and start a communication service (main communication). During the execution of the communication service, the main camera 22 captures the video of the user who sits on the desk 25, and at the same time, the microphone 23 collects sound. The main body 20 acquires real-time video and audio data of the local site from the main camera 22 and the microphone 23, performs an encoding process, and transmits to other sites. Further, the main body 20 receives the video / audio data of the other site, performs a decoding process, and outputs to the display 21 and the speakers 24a, 24b and 24c. Thereby, it becomes possible to mutually output real-time video and audio between a plurality of sites, and a more realistic communication service can be provided.

  In addition, during execution of the communication service, the user operates the remote control 26 to transmit / receive sub-input video such as the sub camera 27 or the notebook PC 28 via the network 10, so that other devices participating in the communication service can communicate with each other. Content can be shared. In content sharing communication, only video data communication is performed, and audio data communication is not performed.

  FIG. 4 is a block diagram showing a hardware configuration of communication apparatus A11 in the present embodiment. As shown in the figure, the communication device A11 includes a control unit 30, an audio encoding unit 31, an audio decoding unit 32, an image encoding unit 33, an image decoding unit 34, a power supply circuit 35, Timer circuit 36, audio processing unit 37, audio output unit 38, audio input unit 39, display processing unit 40, display unit 41, image processing unit 42, image input unit 43, and operation input control unit 44, an operation input unit 45, a communication control unit 46, a transmission / reception circuit 47, a signal confirmation unit 61, and a notification unit 62. Each processing unit is connected to each other through a bus line. If necessary, a hard disk device 48 and a reading device 49 can be connected to the bus line. The hard disk device 48 and the reading device 49 are each connected to a bus line through an interface.

  The control unit 30 includes a CPU 50, a ROM 51, and a RAM 52, and controls the entire communication device A11.

  The CPU 50 may be composed of a single CPU or a plurality of CPUs. The ROM 51 stores a computer program that defines the operation of the CPU 50. The computer program can also be stored in the hard disk device 48. The CPU 50 executes processing defined by the computer program while writing the computer program stored in the ROM 51 or the hard disk device 48 into the RAM 52 as necessary. The RAM 52 also functions as a medium for temporarily storing data generated as the CPU 50 executes processing. The ROM 51 includes a nonvolatile memory and a storage medium that can be written like a flash ROM and can retain stored contents even when the power is turned off. The RAM 52 includes a volatile memory or storage medium that does not retain the stored contents when the power is turned off.

  The audio encoding unit 31 converts the audio data generated by the audio processing unit 37 into encoded audio data by compression encoding using a specific encoding method, and outputs the encoded audio data to the communication control unit 46.

  The audio decoding unit 32 generates reproducible audio data by decoding the encoded audio data output from the communication control unit 46 using a specific decoding method, and outputs the audio data to the audio processing unit 37.

  The image encoding unit 33 converts the image data generated by the image processing unit 42 into encoded image data by compression encoding using a specific encoding method, and outputs the encoded image data to the communication control unit 46.

  The image decoding unit 34 generates reproducible image data by decoding the encoded image data output from the communication control unit 46 using a specific decoding method, and outputs the generated image data to the display processing unit 40.

  When the power key is turned on, the power supply circuit 35 supplies the power received via the battery pack or the adapter to each processing unit, and starts the communication device A11 in an operable state. Further, a plurality of modes of a normal mode and a power saving mode may be provided. In the power saving mode, the required power can be reduced by supplying power only to some of the processing units. For example, in the state of waiting for an incoming call from another base, it operates in the power saving mode for supplying power to the communication control unit 46 and the transmission / reception circuit 47, and switches from the power saving mode to the normal mode when an incoming call is received. Thus, the operation of the other processing unit may be started.

  The timer circuit 36 outputs a timer interrupt signal at a constant cycle.

  The audio processing unit 37 digitally converts the audio signal collected by the one or more audio input units 39, performs synthesis and processing, and outputs the result to the audio encoding unit 31. Clearer audio data can also be generated by a noise cancellation function or the like. Further, when there are a plurality of voice input units 39, the voice processing unit 37 can manage arrangement information of the plurality of voice input units 39 and generate voice data suitable for the arrangement. Further, the voice processing unit 37 performs division and processing of the voice data decoded by the voice decoding unit 32 and outputs the generated voice data to the voice output unit 38. The sound processing unit 37 can manage arrangement information of the plurality of sound output units 38 and generate a plurality of sound data suitable for each sound output unit 38.

  The audio output unit 38 is an output device that outputs audio data. This corresponds to the speakers 24a, 24b and 24c shown in FIG. The audio output unit 38 outputs the audio data generated by the audio processing unit 37.

  The voice input unit 39 is an input device that collects peripheral voices. This corresponds to the microphone 23 shown in FIG. The voice input unit 39 outputs the collected voice signal to the voice processing unit 37.

  The display processing unit 40 divides and processes the image data decoded by the image decoding unit 34 and outputs the generated display data to the display unit 41. Further, the display processing unit 40 manages arrangement information of the plurality of display units 41. As a result, a plurality of display data suitable for each display unit 41 can be generated. In addition, it is possible to display the image of the local site by receiving the image data of the local site from the image processing unit 42 and outputting the image data to the display unit 41.

  The display unit 41 is a device that displays images, characters, and the like. It corresponds to the display 21 shown in FIG. The display unit 41 displays the display data generated by the display processing unit 40 on the screen.

  The image processing unit 42 performs composition processing and processing on the image data output from the one or more image input units 43 and outputs the resultant to the image encoding unit 33. The image processing unit 42 can manage the arrangement information of the plurality of image input units 43 and generate image data suitable for the arrangement. In addition, by outputting the generated image data to the display processing unit 40, it is possible to directly display the image data of the local site on the display unit 41. Furthermore, the image processing unit 42 converts the image data received from the plurality of image input units 43 based on the blocking information notified from the notification unit 62 and the display switching request, such as the display size and location of each image data. Display data is generated by changing arrangement information or synthesizing blocking information.

  The image input unit 43 is a photographing device having a moving image photographing function such as a digital video camera. The image input unit 43 corresponds to the main camera 22, the sub camera 27, and the notebook PC 28 shown in FIG. The image input unit 43 outputs the captured image data to the image processing unit 42 via the signal confirmation unit 61. The image input unit 43 may be a photographing device having only a still image photographing function such as a digital still camera, and may output photographed still image data to the image processing unit 42 at regular intervals.

  The operation input control unit 44 receives the operation input signal output from the operation input unit 45, converts it into corresponding control data, and outputs it to the control unit 30.

  The operation input unit 45 is a device that receives an input instruction from a user and outputs the received input instruction to the operation input control unit 44 as an operation input signal. The operation input unit 45 corresponds to the remote control 26 shown in FIG. The operation input unit 45 may be another operation input device such as a keyboard or a mouse. Further, an input instruction may be received using a gyro function, a sensor function, or the like. The operation input unit 45 includes: 1. main communication establishment signal for requesting establishment of communication for transmitting a main input signal from the main camera 22 to the counterpart device; 2. a communication disconnection signal for requesting disconnection of communication with the counterpart device; By transmitting a sub input signal from the sub camera 27 or the like to the counterpart device, at least a content sharing signal for sharing the sub input signal with the counterpart device is output to the operation input control unit 44 as an operation input signal.

  The communication control unit 46 establishes communication between the other device and itself via the transmission / reception circuit 47, and transmits / receives video / audio data via the network. The communication control unit 46 has, for example, a call connection control (Call Control) function and a data communication control (IP, RTP, RTCP, TCP, etc.) function. Further, the communication control unit 46 receives the encoded audio data encoded by the audio encoding unit 31 and the encoded image data encoded by the image encoding unit 33, and multiplexes them by a predetermined method. The video / audio data to be transmitted is generated.

  In addition, the communication control unit 46 demultiplexes the audio / video data received via the transmission / reception circuit 47 into encoded audio data and encoded image data by performing demultiplexing by a predetermined method, and converts the encoded audio data into The data is output to the audio decoding unit 32, and the encoded image data is output to the image decoding unit 34. When the communication device A11 can be connected to a plurality of different networks, the communication device A11 may include a plurality of communication control units 46 and a plurality of transmission / reception circuits 47 corresponding to each network.

  The transmission / reception circuit 47 is a circuit that performs data transmission / reception via a network. For example, when the network is a wireless line, the transmission data received from the communication control unit 46 is modulated by a predetermined method, and the modulated transmission data is transmitted on a wireless carrier wave. Further, the transmission / reception circuit 47 has a function of receiving a signal in a predetermined frequency band from the high-frequency signal induced in the antenna, demodulating the received signal, and outputting the demodulated signal to the communication control unit 46.

  The hard disk device 48 is a device that writes and reads computer programs or data to and from the built-in hard disk.

  The reading device 49 is a device that reads a computer program or data recorded on a recording medium (for example, a CD, a DVD, or a memory card).

  The signal confirmation unit 61 confirms the presence / absence of an input signal from the image input unit 43 and, when determining that there is no input signal, notifies the notification unit 62 that there is no input signal as a no-signal notification. Further, when the input signal is reacquired after notifying the no-signal notification, the signal confirmation unit 61 notifies the recovery notification indicating that the input signal is reacquired. In addition, the signal confirmation part 61 confirms the presence or absence of each input signal of a main input signal and a sub input signal.

  Based on signal confirmation information such as no-signal notification and restoration notification from the signal confirmation unit 61, the notification unit 62 notifies the image processing unit 42 and the display processing unit 40 with blocking information, a display switching request, and content sharing cancellation. Notify the request. The blocking information is information for notifying that the input signal has been blocked, and the display switching request is information for notifying switching of the display when outputting a plurality of input videos. The content sharing cancellation request is information for notifying that only one input video is to be displayed among a plurality of input videos.

  As described above, the communication device A11 is configured as a computer including a plurality of input / output devices, and can provide communication services with other bases.

  Note that the processing units other than the audio output unit 38, the audio input unit 39, the display unit 41, the image input unit 43, and the operation input unit 45 are provided in the main body 20 shown in FIG. 3, for example.

  Next, the functional configuration of the communication device A11 according to the present embodiment will be described. FIG. 5 is a block diagram illustrating a configuration of the video communication apparatus 100 on the side that transmits the sub input signal according to the present embodiment. The video communication device 100 shown in the figure corresponds to the main body 20 of the communication device A11 on the side that transmits a sub input signal, and is a device that performs processing related to transmission / reception of video signals.

  As shown in FIG. 5, the video communication apparatus 100 includes a main video input unit 101, a sub video input unit 102, a signal detection unit 103, a notification unit 104, a transmission data generation unit 105, a communication unit 106, A display data generation unit 107 and a video output unit 108 are provided.

  The main video input unit 101 is connected to the main camera 22 and acquires a main input signal (main video signal) indicating a main video imaged by the main camera 22. Then, the main video input unit 101 outputs the acquired main input signal to the transmission data generation unit 105. The main video (also referred to as the first main video) is, for example, a video obtained by capturing a user who sits at the desk 25 shown in FIG. The main video input unit 101 is an interface for connecting the image input unit 43 and the image processing unit 42 shown in FIG.

  The sub video input unit 102 is connected to the sub camera 27 and acquires a sub input signal (sub video signal) indicating a sub video imaged by the sub camera 27. Then, the sub video input unit 102 outputs the acquired sub input signal to the signal detection unit 103, the transmission data generation unit 105, and the display data generation unit 107. The sub video is, for example, a video obtained by capturing an image of a user's hand. Note that the sub video input unit 102 may be connected to another input device such as the notebook PC 28 to acquire a video signal input from the other input device as a sub input signal. Further, the sub video input unit 102 is an interface for connecting the image input unit 43 and the signal confirmation unit 61 shown in FIG.

  The signal detection unit 103 corresponds to the signal confirmation unit 61 shown in FIG. 4 and detects whether or not a sub input signal is acquired by the sub video input unit 102, that is, the presence or absence of the sub input signal. When the sub input signal is not acquired, the signal detection unit 103 outputs a no-signal notification indicating that the sub input signal is not acquired to the notification unit 104. In addition, when the sub input signal is reacquired, the notification unit 104 is notified of a recovery notification indicating that.

  The notification unit 104 corresponds to the notification unit 62 shown in FIG. 4 and notifies the transmission data generation unit 105 and the display data generation unit 107 according to whether or not a sub input signal is input. Specifically, the notification unit 104 includes a blocking information notification unit 141 and a display switching request notification unit 142.

  The blocking information notifying unit 141 indicates blocking information indicating that the sub input signal is blocked when the signal detecting unit 103 detects that the sub input signal has not been continuously acquired for a predetermined first period. Is generated. Then, the generated cutoff information is notified to the transmission data generation unit 105, the display data generation unit 107, and the display switching request notification unit 142.

  Specifically, the cutoff information notification unit 141 starts a cutoff information notification timer (not shown) at the same time that the no-signal notification is received from the signal detection unit 103. And when not receiving the recovery notification which shows that the sub input signal was acquired from the signal detection part 103 before the 1st period passes, interruption | blocking information is produced | generated and notified.

  The first period may be any period such as 5 seconds, 10 seconds, or 1 minute. The first period may be 0 seconds. That is, at the same time that the no-signal notification is received, the cutoff information may be generated and notified. The cutoff information notification timer is reset at the same time as notification of cutoff information or at the same time as receiving a recovery notification from the signal detection unit 103.

  The display switching request notifying unit 142 sends a display switching request to the transmission data generating unit 105 after the blocking information is generated by the blocking information notifying unit 141, that is, after a predetermined second period has elapsed after receiving the blocking information. This is notified to the display data generation unit 107. The display switching request is an instruction to switch display between the main video and the sub video.

  Specifically, the display switching request notifying unit 142 activates a display switching timer (not shown) at the same time that the blocking information is received. When the restoration notification is not received from the signal detection unit 103 until the second period elapses, a display switching request is notified.

  The second period may be any period such as 5 seconds, 10 seconds, or 1 minute. The second period may be 0 seconds. In other words, the display switching request may be notified at the same time that the no-signal notification is received. In addition, the display switching timer is reset at the same time that the display switching request is notified or the recovery notification is received from the signal detection unit 103.

  Further, the display switching request notification unit 142 notifies the transmission data generation unit 105 and the display data generation unit 107 of a content sharing cancellation request after a predetermined third period has elapsed since the display switching request was notified.

  The transmission data generation unit 105 corresponds to the image encoding unit 33 and the image processing unit 42 illustrated in FIG. 4 and generates transmission data including at least one of the main video and the sub video. At this time, when blocking information is notified from the blocking information notification unit 141, the transmission data generation unit 105 generates transmission data including the notified blocking information. Specifically, the transmission data is generated so that the blocking information is displayed as character information on the display unit of the communication device B12. The generated transmission data is output to the video transmission unit 161.

  When the display switching request is notified from the display switching request notifying unit 142, the transmission data generating unit 105 generates transmission data so that the main video is displayed larger than the sub video. Furthermore, when a content sharing cancellation request is notified from the display switching request notification unit 142, transmission data is generated so that the main video is displayed on the full screen.

  The communication unit 106 corresponds to the communication control unit 46 and the transmission / reception circuit 47 illustrated in FIG. 4, establishes communication with other devices, and transmits / receives data using the established communication. As illustrated in FIG. 5, the communication unit 106 includes a video transmission unit 161 and a video reception unit 162.

  The video transmission unit 161 corresponds to the communication control unit 46 and the transmission / reception circuit 47 illustrated in FIG. 4, and transmits the transmission data generated by the transmission data generation unit 105 to other devices via the network 10.

  The video receiving unit 162 corresponds to the communication control unit 46 and the transmission / reception circuit 47 shown in FIG. 4 and receives received data from other devices via the network 10. The received data includes, for example, an image of the other user captured by a camera provided in another device.

  The display data generation unit 107 corresponds to the image decoding unit 34 and the display processing unit 40 shown in FIG. 4, and generates display data including a video (also referred to as a second main video) included in the received data and a sub video. To do. At this time, when the blocking information is notified from the blocking information notification unit 141, the display data generation unit 107 generates display data including the notified blocking information and the video included in the received data. Specifically, display data is generated so that the blocking information is displayed on the display 21 as character information. The generated display data is output to the video output unit 108.

  When the display switching request is notified from the display switching request notifying unit 142, the display data generating unit 107 generates display data so that the video included in the received data is displayed larger than the sub video. Furthermore, when a content sharing cancellation request is notified from the display switching request notification unit 142, transmission data is generated so that the video included in the received data is displayed on the full screen.

  The video output unit 108 outputs the display data generated by the display data generation unit 107 to the display 21. The video output unit 108 is an interface that connects the display processing unit 40 and the display unit 41 shown in FIG.

  Next, the operation of the communication device according to the present embodiment will be described. Specifically, a display switching method between an input signal from the main camera 22 (hereinafter referred to as a main input signal) and an input signal from the sub camera 27 (hereinafter referred to as a sub input signal), and notification of cutoff information Will be described in detail. An example in which main communication is established between the communication device A11 and the communication device B12 and content of the sub video by the sub camera 27 of the communication A11 is shared will be described with reference to FIGS.

  FIG. 6 is a flowchart showing an operation for establishing main communication and an operation for starting sharing of content in the communication apparatus A11.

  FIG. 7 is a flowchart showing an operation when the sub input signal in the communication device A11 is interrupted.

  FIG. 8 is a diagram illustrating image transitions displayed on the display unit of the communication device B12. The display unit of the communication device B12 displays data received from the communication device A11 via the communication network.

  In the present embodiment, the transmission data generation unit 105 and the display data generation unit 107 define a first area and a second area having a size equal to or smaller than the first area in the display unit, and a main area in each area. Transmission data or display data is generated so that a video and a sub video are displayed. Here, for the sake of simplicity, a parent screen (first area) and a child screen (second area) are defined, the parent screen is displayed as a full screen, and the child screen is arranged small in the upper right of the full screen display. .

  FIG. 9 is a diagram illustrating image transition displayed on the display unit 41 of the communication device A11.

  First, operations for establishing main communication and starting sharing of content will be described with reference to FIGS. 6, 8, and 9.

  First, main communication is established between the communication device A11 and the communication device B12 using the main input signal (S101).

  Specifically, in the communication apparatus A11, when the main camera 22 is turned on, the signal confirmation unit 61 receives the main input signal. Thereafter, when the conference person makes a main communication establishment request from the remote controller 26, the operation input control unit 44 transmits a main communication establishment signal to the communication control unit 46, and the communication control unit 46 performs main communication with the communication device B12. Establish. Then, by performing data communication control, transmission / reception of image data generated from the main input signals of both communication apparatuses is started.

  At this time, in the communication device A11, the transmission data generation unit 105 (image processing unit 42) generates transmission data so that the main input signal is displayed as the main screen on the display unit of the communication device B12. That is, the transmission data generation unit 105 generates transmission data so that the main video included in the main input signal is displayed on the full screen before content sharing is started. Thereby, the video 201 shown in FIG. 8 is displayed on the display unit of the communication device B12. As shown in FIG. 8, the video 201 is a main video (here, three participants) captured by the main camera 22 of the communication device A11.

  At this time, in the communication device A11, the display data generation unit 107 (display processing unit 40) generates display data to be displayed on the display 21 using the video signal received from the communication device B12. Similar to the transmission data generation unit 105, the display data generation unit 107 displays the display data so that the video included in the video signal received from the communication device B12 is displayed on the full screen before content sharing is started. Generate. As a result, the video 301 shown in FIG. 9 is displayed on the display 21. As shown in FIG. 9, the video 301 is a main video (here, one participant) captured by the main camera of the communication device B12.

  The call connection control and data communication control are not limited as long as image data generated from the input video can be transmitted and received.

  The communication device A11 shares content with the communication device B12 using the sub camera 27 or the like at a timing required by the conference person. In the present embodiment, content sharing is performed using the main communication established first.

  In preparation for content sharing, the signal detection unit 103 (signal confirmation unit 61) detects whether or not a sub input signal is acquired (S102). When the sub input signal is detected (Yes in S102), that is, when the conference person connects the sub camera 27 to the main body 20 and turns on the power, the signal detection unit 103 (signal confirmation unit 61) Upon receiving the signal, the communication device A11 starts content sharing (S103).

  Specifically, the transmission data generation unit 105 determines the display position of the main input signal and the sub input signal, and generates transmission data so that the main video and the sub video are displayed at the determined display position. For example, when the conference person makes a content sharing request using the remote control 26 or the like, the operation input control unit 44 receives the content sharing request and notifies the image processing unit 42 of the content sharing request. The image processing unit 42 sets the sub input signal to the parent screen and sets the main input signal to the child screen. That is, content sharing is performed on the main communication path established first by the image processing unit 42 setting the display position. Then, the video transmission unit 161 (communication control unit 46) transmits the transmission data generated by the image processing unit 42 to the communication device B12.

  As a result, on the display unit of the communication device B12, the sub video is displayed on the main screen 212 and the main video is displayed on the sub screen 213, as indicated by the video 211 in FIG.

  At this time, the sub video may be displayed on the display unit 41 of the communication device A11 as necessary. For example, the display data generation unit 107 (display processing unit 40) displays the sub video on the main screen 312 and the video received from the communication device B12 on the sub screen 313 as shown in the video 311 in FIG. Thus, display data is generated.

  In this way, content sharing is started between the communication device A11 and the communication device B12.

  Next, an operation when the sub input signal is interrupted for some reason will be described with reference to FIGS. Since the sub video is already shared, the video 211 shown in FIG. 8 is displayed on the display unit of the communication device B12, and the video 311 shown in FIG. 9 is displayed on the display unit 41 of the communication device A11. Suppose that it is displayed.

  After the start of content sharing, the signal detection unit 103 (signal confirmation unit 61) always confirms whether or not the sub input signal is not blocked (S201).

  When the sub input signal is detected to be blocked due to a cable disconnection from the main body 20 of the sub camera 27 or the like (Yes in S201), the signal detection unit 103 (signal confirmation unit 61) blocks no signal notification. The information notification unit 141 is notified (S202).

  Since the sub input signal is not input, the sub video is not displayed on the main screen 222 and the main video is displayed only on the sub screen 223 on the display unit of the communication device B12 as shown in the video 221 of FIG. Is done. Further, on the display 21 (display unit 41) of the communication device A11, as shown in the video 321 in FIG. 9, the sub video is not displayed on the parent screen 322, and the main video is displayed only on the child screen 323.

  The blocking information notification unit 141 starts the blocking information notification timer simultaneously with receiving the no-signal notification. Until the first period elapses (No in S204), the signal detection unit 103 detects whether or not the input of the sub input signal is restored, that is, whether or not the sub input signal is input again ( S203). When the sub input signal is input again (Yes in S203), the signal detection unit 103 outputs a recovery notification to the cutoff information notification unit 141, and continues to check whether there is a sub input signal (S201).

  If the sub-input signal cannot be detected again within a predetermined time, the blocking information notification timer receives a sub-input to the transmission data generation unit 105 (image processing unit 42) and the display data generation unit 107 (display processing unit 40). This is a timer for notifying blocking information indicating that blocking has occurred. That is, it is a timer for determining a period until notifying the conference person that the sub input is blocked, and the fixed time may be zero seconds.

  When the sub-input signal cannot be detected within the first period after the blocking information notification timer is activated (Yes in S204), the blocking information notification unit 141 and the transmission data generation unit 105 (image processing unit 42) and display data generation Blocking information is notified to the unit 107 (display processing unit 40) and the display switching request notification unit 142 (S205).

  The transmission data generation unit 105 (image processing unit 42) receives the blocking information, generates transmission data including that the sub input signal is blocked, and transmits the generated transmission data to the video transmission unit 161 (communication control unit). 46). That is, the transmission data generation unit 105 generates transmission data so that the blocking information is displayed in the third area of the display unit of the communication device B12. Thereby, on the display unit of the communication device B12, as shown in the video 231 of FIG. 8, the sub video is not displayed on the parent screen 232, but the blocking information 234 is displayed. A main video is displayed on the sub-screen 233.

  Similarly, the display data generation unit 107 (display processing unit 40) generates display data including that the sub input signal is blocked, and outputs the generated display data to the display 21 (display unit 41). That is, the display data generation unit 107 generates display data so that the blocking information is displayed in the third area of the display 21. As a result, on the display 21 (display unit 41) of the communication device A11, as shown in the video 331 of FIG. 9, the sub video is not displayed on the parent screen 332, but the blocking information 334 is displayed. In addition, the main video is displayed on the sub-screen 333.

  The display switching request notification unit 142 starts the display switching timer at the same time as receiving the blocking information. Until the second period elapses (No in S207), the signal detection unit 103 detects whether or not the input of the sub input signal is restored, that is, whether or not the sub input signal is input again ( S206). When the sub input signal is input again (Yes in S206), the signal detection unit 103 outputs a recovery notification to the display switching request notification unit 142, and continues to check whether there is a sub input signal (S201).

  When the sub-input signal cannot be detected within a certain time, the display switching timer instructs the transmission data generation unit 105 (image processing unit 42) and the display data generation unit 107 (display processing unit 40) to display the parent screen and the child screen. This is a timer for transmitting a display switching request for switching the display position of the screen. The display switching timer is configured such that when the sub input signal is the notebook PC 28, when the display position is switched without the timer when the notebook PC is once disconnected and another notebook PC is connected. There is an effect of reducing the discomfort of the conference participant due to the flickering of the screen.

  If the sub input signal cannot be detected within the second period after the display switching timer is activated (Yes in S207), the display switching request notification unit 142 and the transmission data generation unit 105 (image processing unit 42) and display data generation A display switching request is notified to the unit 107 (display processing unit 40) (S208).

  The transmission data generation unit 105 (image processing unit 42) receives the display switching request, sets the main screen as the main video included in the main input signal, and sets the sub-screen as the sub video included in the sub input signal. That is, when the display switching request is notified, the transmission data generation unit 105 generates transmission data so that the main video is displayed on the parent screen. At this time, since the sub input signal has not been acquired, an image (such as a black screen) indicating a no-signal state is displayed on the child screen. As a result, the main video is displayed on the main screen 242 on the display unit of the communication device B12 as shown in the video 241 in FIG. Further, an image without a sub input signal is displayed on the sub-screen 243.

  Similarly, the display data generation unit 107 (display processing unit 40) of the communication device A11 sets the main video including the main input signal on the parent screen and the sub video including the sub input signal on the child screen. That is, the display data generation unit 107 also generates display data so that the main video is displayed on the parent screen when the display switching request is notified. At this time, since the sub input signal has not been acquired, an image (such as a black screen) indicating a no-signal state is displayed on the child screen. As a result, as shown in the video 341 in FIG. 9, the main video is displayed on the main screen 342 and the video without the sub input signal is displayed on the sub-screen 343 on the display 21 (display unit 41). .

  Thereafter, the display switching request notification unit 142 notifies the display switching request and starts the content sharing cancellation timer at the same time. Until the third period elapses (No in S210), the signal detection unit 103 detects whether or not the input of the sub input signal is restored, that is, whether or not the sub input signal is input again ( S209). When the sub input signal is input again (Yes in S209), the signal detection unit 103 outputs a recovery notification to the display switching request notification unit 142, and continues to check whether there is a sub input signal (S201).

  The content sharing cancellation timer is a timer for canceling the display of the sub input signal when the sub input signal is not detected within a certain period, that is, for preventing the child screen from being displayed.

  If the sub input signal cannot be detected within the third period after the content sharing cancellation timer is activated (Yes in S210), the display switching request notification unit 142 sends a content sharing cancellation request to the transmission data generation unit 105 (image processing unit). 42) and the display data generation unit 107 (display processing unit 40) (S211).

  The transmission data generation unit 105 (image processing unit 42) generates transmission data so that only the parent screen is displayed by deleting the child screen. Thereby, only the main video is displayed on the display unit of the communication device B12 as shown in the video 201 of FIG.

  Similarly, the display data generation unit 107 (display processing unit 40) generates display data so that only the parent screen is displayed by deleting the child screen. Thus, only the main video received from the communication device B12 is displayed on the display 21 (display unit 41) of the communication device A11 as shown in the video 301 of FIG.

  As described above, the video communication apparatus 100 (communication apparatus A11) according to the present embodiment checks the presence / absence of the sub input signal, and when the sub input signal is blocked, the display unit of the own apparatus and the partner Transmission data and display data are generated so as to be displayed on the display unit of the apparatus.

  Thereby, in the electronic conference system using the video communication apparatus 100 of the present embodiment, even when a black screen (screen indicating that there is no signal) is displayed on the display unit, the blocking information is displayed on the display unit. Is displayed. Therefore, the participant can easily confirm that the sub input signal is blocked.

  At present, electronic conferencing systems are mainly used for conferencing purposes, but in the future, they will be widely used in homes, etc., and will be used for communication between grandparents' homes and homes of children and grandchildren located far away. There is a possibility that. At that time, since it is assumed that a user who is not familiar with the operation of the electronic device is used, it is required to simplify the user operation and clarify the state of the device.

  Therefore, the video communication apparatus 100 according to the present embodiment displays the sub video from the parent screen to the child screen without any operation by the participant if a predetermined period (second period) elapses after the blocking information is notified. Change automatically. Therefore, even if the participant is not familiar with the operation of the present apparatus, the electronic conference can be continued smoothly.

  Note that the communication device B12 on the side receiving the sub input signal may be a device having call connection control, data communication control, and a content sharing function. Specifically, the communication device B12 includes a main video input unit 101, a transmission data generation unit 105, a video transmission unit 161, a video reception unit 162, a display data generation unit 107, and a video output unit illustrated in FIG. And a processing unit corresponding to 108 may be provided. At this time, the transmission data generation unit 105 and the display data generation unit 107 included in the communication device B12 do not need a function of changing the display position.

  Before making a content sharing cancellation request, cancellation information 444 indicating that content sharing cancellation is awaiting may be displayed as shown in an image 441 in FIG. The video 441 corresponds to the video 241 in FIG.

  Specifically, at the same time as the transmission data generation unit 105 switches the display, the transmission data generation unit 105 generates transmission data so that the release information 444 is displayed. Similarly, the display data generation unit 107 generates display data so that the release information 444 is displayed at the same time as the display is switched. The release information 444 is generated by, for example, the display switching request notification unit 142 and output to the transmission data generation unit 105 and the display data generation unit 107.

  Further, in the present embodiment, the description has been given using the parent screen and the child screen. However, the main input signal and the sub input signal may be switched using only the parent screen without using the child screen. . FIG. 11 is a diagram illustrating an example of different image transitions displayed on the display unit of the communication device B12.

  As shown in the figure, before the content sharing is started, the display unit of the communication device B12 displays a video 501 (corresponding to the video 201 in FIG. 8). The video 501 is a main video acquired from the main camera 22 of the communication device A11.

  When content sharing is started (S103 in FIG. 6), the display unit displays a video 511 (corresponding to the video 211 in FIG. 8). The video 511 is a sub video acquired from the sub camera 27 of the communication device A11.

  As described above, when content sharing is started, the display unit of the communication device B12 may display only the sub video without displaying the main video. That is, the transmission data generation unit 105 of the communication device A11 may generate transmission data so that only the sub video is displayed on the display unit of the communication device B12.

  When the sub input signal is cut off (Yes in S201 in FIG. 7), the display unit displays the video 521 (corresponding to the video 221 in FIG. 8). The video 521 is a black screen indicating that there is no signal.

  After the first period has elapsed (Yes in S204 in FIG. 7), the display unit displays the video 531 (corresponding to the video 231 or 241 in FIG. 8). Video 531 is a video in which blocking information 534 is displayed on a black screen or the like indicating no signal. That is, the transmission data generation unit 105 of the communication device A11 generates transmission data so that the blocking information is displayed on the display unit of the communication device B12.

  Further, when the second period elapses (Yes in S206 in FIG. 7), the display unit displays a video 501 (corresponding to the video 201 in FIG. 8). That is, when the display switching request is notified, the transmission data generation unit 105 of the communication device A11 generates transmission data so that the main video is displayed on the display unit of the communication device B12.

  As described above, when the main video and the sub video are switched and displayed without using the main screen and the sub screen, especially for the participant on the communication device B12 side, when the sub video is blocked, It is difficult to identify the cause of this blockage. This is because a black screen like the video 521 is displayed on the display unit regardless of whether the communication of the main video is cut off or only the sub video is cut off.

  In the present embodiment, when the sub input signal is blocked, the block information 534 is displayed as shown in the video 531, so that the participant on the communication device B12 side can easily block the sub input signal. Can be recognized.

  In this embodiment, the sub input signal is cut off as a cable disconnection. However, the sub input signal is cut off of the battery of the sub camera 27 (image input unit 43), the power is turned off, and the D terminal signal is cut off. The HDMI signal may be disconnected. In particular, when a signal stop command is transmitted from the sub-camera 27 in CEC (Consumer Electronics Control) control on HDMI, etc., it is determined that the sub-input signal is blocked by the conference, and the blocking of the sub-input signal is detected (S201 (Yes), the main video may be immediately displayed on the main screen. That is, the display unit may change the display from the video 221 to the video 241 or the video 201.

  Specifically, the signal detection unit 103 (signal confirmation unit 61) receives a signal stop command from the sub camera 27, and outputs the received signal stop command to the notification unit 104 (notification unit 62). When receiving the signal stop command, the notification unit 104 outputs a display switching request to the transmission data generation unit 105 and the display data generation unit 107 without starting the display switching timer.

  In this way, the notification unit 104 may output a display switching request to the transmission data generation unit 105 and the display data generation unit 107 when receiving a stop command indicating that the sub input signal is stopped from the user. Thereby, when the user intentionally stops the sub input signal, the main video can be displayed in a large size without having to wait for the second period, so that the electronic conference can be continued more smoothly.

  In this embodiment, the blocking information notification timer, the display switching timer, and the content sharing cancellation timer are used. However, the notification unit 104 can immediately block the blocking information, the display switching request, or the content sharing without starting each timer. A cancellation request may be notified. That is, the first period, the second period, and the third period described above may be 0 seconds.

  In the present embodiment, the display position of the main video and the sub video has been described using only the parent screen and the parent screen + slave screen. It may be a screen. At this time, the size of each screen may be the same size. Moreover, although the notification location of the cutoff information has been described as the lower side of the screen, it may be anywhere as long as it is displayed on the display 21 (display unit 41).

  Further, in the present embodiment, the blocking information notification method is performed on the main screen, but the conference person may be notified by voice. That is, the transmission data generation unit 105 and the display data generation unit 107 may generate transmission data or display data so that the blocking information is output as sound.

  Moreover, you may use the information display part which displays the information which should be shown to users, such as interruption | blocking information, such as not a main screen but the liquid crystal screen with which the main body 20 is equipped, or LED (Light Emitting Diode).

  When the interruption of the sub input signal of the communication device A11 is detected, the main camera 22 of the communication device A11 detects the main body of the sub camera 27 from the video, performs zoom photography, and displays it on the display unit 41 of the communication device A11. By outputting, it may be used to identify the cause of the interruption.

  Further, in the present embodiment, the signal confirmation unit 61 of the communication device A11 has been described as detecting the blockage of the signal. However, the display processing unit of the communication device B12 has an image analysis function, and the sub input signal video is displayed. It may be determined that the sub input signal is blocked when the screen is black, and the same block information notification and display switching control as in the present embodiment may be performed.

(Embodiment 2)
In the first embodiment, the content sharing by the sub camera 27 is performed by the same communication as the communication by the main camera 22. However, in the present embodiment, the content sharing communication is performed separately from the communication by the main camera 22. And share the content with the established content sharing communication. At this time, the communication device A11 only notifies the communication device B12 of a no-signal notification, and the communication device B12 carries out notification of blocking information, switching processing of the display position, and the like. Accordingly, first, the functional configuration of the communication device A11 will be described below.

  FIG. 12 is a block diagram illustrating a functional configuration of the video communication apparatus 600 on the side that transmits the sub input signal according to the present embodiment. The video communication device 600 corresponds to the communication device A11 and is a device that performs processing related to transmission / reception of video signals. Compared with the video communication device 100 of FIG. 5, the video communication device 600 of FIG. 5 includes a signal detection unit 603 instead of the signal detection unit 103, a notification unit 604 of the notification unit 104, and a transmission data generation unit 105. The transmission data generation unit 605 is provided instead of the communication unit 106, and the communication unit 606 is provided instead of the communication unit 106. In the following, description of the same points as in the first embodiment will be omitted, and different points will be mainly described.

  Similarly to the signal detection unit 103, the signal detection unit 603 detects whether or not a sub input signal is acquired by the sub video input unit 102. The signal detection unit 603 notifies the notification unit 604 and the communication unit 606 of a no-signal notification and a recovery notification according to the detection result.

  The notification unit 604 notifies the display data generation unit 107 of the cutoff information and the display switching request depending on whether or not the sub input signal is acquired. As illustrated in FIG. 12, the notification unit 604 includes a blocking information notification unit 641 and a display switching request notification unit 642.

  The blocking information notification unit 641 performs the same processing as the blocking information notification unit 141. However, the difference is that only the display data generation unit 107 is notified of the blocking information.

  The display switching request notification unit 642 performs the same processing as the display switching request notification unit 142. However, the difference is that only the display data generation unit 107 is notified of the display switching request.

  The transmission data generation unit 605 generates main transmission data including the first main video acquired by the main video input unit 101 and sub transmission data including the sub video acquired by the sub video input unit 102. The generated main transmission data and sub transmission data are output to the video transmission unit 661 included in the communication unit 606. The first main video is, for example, a video taken by the main camera 22 included in the communication device A11, such as a user.

  The communication unit 606 establishes two communications, a main communication for main transmission data and a sub-communication for sub-transmission data, and transmits and receives data using the established two communications. As illustrated in FIG. 12, the communication unit 606 includes a video transmission unit 661 and a video reception unit 662.

  The video transmission unit 661 transmits main transmission data to another video communication device (communication device B12) using main communication. Furthermore, sub transmission data is transmitted to communication apparatus B12 using sub communication. In addition, when no signal notification or recovery notification is notified from the signal detection unit 603, the no signal notification or recovery notification is transmitted to the communication device B12 using main communication or sub communication.

  The video receiver 662 receives main received data from another video communication device (communication device B12) using main communication. The main reception data includes, for example, an image of the other user captured by a main camera provided in the communication device B12.

  As described above, the video communication apparatus 600 (communication apparatus A11) of the present embodiment transmits the presence / absence of the sub input signal to the communication apparatus B12.

  Next, the functional configuration of the communication device B12 will be described.

  FIG. 13 is a block diagram illustrating a functional configuration of the video communication apparatus 700 on the side receiving the sub input signal according to the present embodiment. The video communication device 700 corresponds to the communication device B12, and is a device that performs processing related to transmission / reception of video signals. As shown in the figure, the video communication apparatus 700 includes a main video input unit 701, a notification unit 704, a transmission data generation unit 705, a communication unit 706, a display data generation unit 707, and a video output unit 708. Prepare.

  The main video input unit 701 is connected to the main camera 722 and acquires a main input signal indicating a second main video imaged by the main camera 722. Then, the acquired main input signal is output to transmission data generation section 705. Note that the second main video is a video taken by the main camera 722 provided in the communication device B12 to photograph a user or the like, and is displayed on the display 21 provided in the communication device A11.

  The notification unit 704 notifies the display data generation unit 707 of blocking information, a display switching request, and the like based on the no-signal notification transmitted from the video communication device 600 (communication device A11) by the video reception unit 762. As illustrated in FIG. 13, the notification unit 704 includes a blocking information notification unit 741 and a display switching request notification unit 742.

  The blocking information notification unit 741 generates blocking information when a no-signal notification is received from the video reception unit 762. In addition, the process which produces | generates interruption | blocking information is the same as that of the interruption | blocking information notification part 141 or 641. Then, the generated cutoff information is notified to the display data generation unit 707 and the display switching request notification unit 742. Note that the blocking information is information indicating that a sub input signal from the sub camera 27 and the like included in the communication device A11 is not input, as in the first embodiment.

  The display switching request notifying unit 742 notifies the display data generating unit 707 of the display switching request after a predetermined second period has elapsed since receiving the blocking information from the blocking information notifying unit 741. The process of notifying the display switching request is the same as that of the display switching request notifying unit 142 or 642.

  The transmission data generation unit 705 generates main reception data including the second main video. The generated main reception data is output to the video transmission unit 761 provided in the communication unit 706.

  The communication unit 706 transmits and receives data using two communications, main communication and sub-communication. As illustrated in FIG. 13, the communication unit 706 includes a video transmission unit 761 and a video reception unit 762.

  The video transmission unit 761 transmits main reception data to the video communication device 600 (communication device A11) using main communication.

  The video receiver 762 receives main transmission data from the video communication device 600 (communication device A11) using main communication. Furthermore, sub transmission data is received from communication apparatus A11 using sub communication. Further, it receives a no-signal notification or a recovery notification, and outputs the received no-signal notification or recovery notification to the blocking information notification unit 741.

  The display data generation unit 707 generates display data including the first main video and the sub video. At this time, when the blocking information is notified from the blocking information notification unit 741, the display data generation unit 707 generates display data including the notified blocking information and the first video. The generated display data is output to the video output unit 708. When the display switching request is notified from the display switching request notification unit 742, the display data generation unit 707 generates display data so that the video included in the received data is displayed larger than the sub video.

  The video output unit 708 outputs the display data generated by the display data generation unit 707 to the display 721.

  As described above, the video communication apparatus 700 (communication apparatus B12) of the present embodiment receives information (no signal notification and recovery notification) regarding the presence or absence of the sub input signal from the communication apparatus A11, and based on the information, Notification of blocking information, display position switching processing, and the like are performed. That is, unlike in the first embodiment, the communication device B12 on the side that receives the sub input signal determines at what timing the blocking information is displayed on the display 721 and at what timing the video display is changed.

  Next, the operation of the communication device according to the present embodiment will be described. An image displayed on the display 721 of the communication device B12 will be described with reference to FIG.

  FIG. 14 is a flowchart showing an operation of establishing main communication and sub-communication and an operation of starting content sharing in the communication device A11.

  First, main communication is established between the communication device A11 and the communication device B12 using the main input signal (S301). Note that the specific processing is the same as that in the first embodiment (S101 in FIG. 7), and thus the description thereof is omitted. As a result, the video 201 shown in FIG. 8 is displayed on the display 721. A video 201 is a video taken by the main camera 22 of the communication apparatus A.

  Next, in preparation for content sharing, the signal detection unit 103 detects whether or not a sub-input signal is input (S302). When the sub input signal is detected (Yes in S302), sub communication for communicating the sub input signal is established between the communication device A11 and the communication device B12 (S302). As a result, the video 211 shown in FIG. 8 is displayed on the display 721. In other words, the sub video captured by the sub camera 27 is displayed on the main screen 212, and the main video captured by the main camera 22 is displayed on the sub screen 213.

  Thus, in the present embodiment, two communications are used: main communication for main input signal and sub communication for sub input signal. Therefore, the compositing process for displaying both the main video included in the main input signal and the sub video included in the sub input signal is executed by the communication device B12 on the side receiving the sub input signal.

  Hereinafter, among the operations of the communication device B12 (video communication device 700), an operation when the sub input signal is not acquired by the signal detection unit 603 will be described.

  FIG. 15 is a flowchart illustrating an operation when the sub input signal is blocked in the communication device B12. The process shown in the figure is almost the same as the operation of communication apparatus A11 described in the first embodiment. The same reference numerals are assigned to the same processing steps.

  First, when receiving no signal notification from the communication device A11 (Yes in S401), the video reception unit 762 outputs the received no signal notification to the blocking information notification unit 741. At this time, since the sub input signal is not received, the video 221 in FIG. 8 is displayed on the display 721.

  The blocking information notification unit 741 starts the blocking information notification timer simultaneously with receiving the no-signal notification. When the video reception unit 762 receives a recovery notification from the communication device A11 until the first period elapses (No in S204), that is, when the sub transmission data is received again (Yes in S203), the display data The generation unit 707 generates display data including the sub video and the main video included in the re-received sub transmission data. At this time, the video 221 shown in FIG. 8 is displayed on the display 721.

  When the first period has elapsed without receiving a recovery notification after starting the blocking information notification timer (Yes in S204), the blocking information notification unit 741 generates blocking information, and displays the display switching request notification unit 742 and the display data. This is notified to the generation unit 707 (S205). The display data generation unit 707 generates display data including the received blocking information and the main video. As a result, the video 231 shown in FIG. 8 is displayed on the display 721.

  The display switching request notifying unit 742 starts the display switching timer at the same time that the blocking information is received. Until the second period elapses (No in S207), when the video reception unit 762 receives a recovery notification from the communication device A11, that is, when the sub transmission data is received again (Yes in S206), the display data The generation unit 707 generates display data including the sub video and the main video included in the re-received sub transmission data. At this time, the video 221 shown in FIG. 8 is displayed on the display 721.

  When the second period has elapsed without receiving a recovery notification after starting the display switching timer (Yes in S207), the display switching request notification unit 742 notifies the display data generation unit 707 of the display switching request. The display data generation unit 707 receives the display switching request, sets the main screen as the main video included in the main input signal, and sets the sub-screen as the sub video included in the sub input signal (S208). Thereby, the video 241 in FIG. 8 is displayed on the display 721.

  Then, the display switching request notification unit 142 notifies the display switching request and starts the content sharing cancellation timer at the same time. When the video reception unit 762 receives a recovery notification from the communication device A11 until the third period elapses (No in S210), that is, when the sub transmission data is received again (Yes in S209), the display data The generation unit 707 generates display data including the sub video and the main video included in the re-received sub transmission data. At this time, the video 221 shown in FIG. 8 is displayed on the display 721.

  When the sub input signal cannot be detected within the third period after the content sharing cancellation timer is activated (Yes in S210), the display switching request notification unit 742 notifies the display data generation unit 707 of the content sharing cancellation request ( S211). The display data generation unit 707 generates display data so that only the parent screen is displayed by deleting the child screen. As a result, the video 201 in FIG. 8 is displayed on the display 721.

  As described above, the video communication system (electronic conference system) using the video communication apparatus 600 (communication apparatus A11) and the video communication apparatus 700 (communication apparatus B12) according to the present embodiment has two main communication and sub-communication systems. Video data is transmitted and received using two communications. At this time, the video communication device 700 on the side receiving the sub video displays the blocking information on the display 721 when the sub video is not received according to whether or not the sub video is received.

  Thereby, even if it is a case where two communication, main communication and sub communication, is used, interruption | blocking information is displayed on both display parts. Therefore, the participant participating in the electronic conference system can easily confirm that the sub input signal is blocked by looking at the blocking information displayed on the display 721.

  Further, as in the first embodiment, the video communication apparatuses 600 and 700 according to the present embodiment automatically display the sub video from the parent screen to the child screen after notifying the blocking information without any participant operation. change. Therefore, even if the participant is not familiar with the operation of the present apparatus, the electronic conference can be continued smoothly.

(Embodiment 3)
In the present embodiment, in the same manner as in the second embodiment, notification of blocking information is performed in a manner different from that in the second embodiment when establishing sub-communication for content sharing separately from the main communication by the main camera 22. And notification of a display switching request.

  FIG. 16 is a block diagram illustrating a functional configuration of the video communication apparatus 800 on the side that transmits the sub input signal according to the present embodiment. The video communication device 800 corresponds to the communication device A11 and is a device that performs processing related to transmission / reception of video signals. The video communication apparatus 800 shown in the figure is different from the video communication apparatus in FIG. 12 in that a notification unit 804 is provided instead of the notification unit 604 and a communication unit 806 is provided instead of the communication unit 606. In the following, description of the same points as in the second embodiment will be omitted, and different points will be mainly described.

  The notification unit 804 notifies the display data generation unit 107 and the video transmission unit 861 included in the communication unit 806 of the cutoff information and the display switching request depending on whether or not the sub input signal is acquired. As illustrated in FIG. 16, the notification unit 804 includes a blocking information notification unit 841 and a display switching request notification unit 842.

  The blocking information notification unit 841 performs the same process as the blocking information notification unit 641. However, the difference is that not only the display data generation unit 107 but also the video transmission unit 861 included in the communication unit 806 is notified of the cutoff information.

  The display switching request notification unit 842 performs the same processing as the display switching request notification unit 642. However, the difference is that not only the display data generation unit 107 but also the video transmission unit 861 is notified of the display switching request.

  Similar to the communication unit 606, the communication unit 806 establishes two communications, a main communication for main transmission data and a sub communication for sub transmission data, and transmits and receives data using the two established communications. Do. As illustrated in FIG. 16, the communication unit 806 includes a video transmission unit 861 and a video reception unit 662. Note that the video receiving unit 662 performs the same processing as in the second embodiment, and a description thereof will be omitted here.

  The video transmission unit 861 transmits main reception data to another video communication device (communication device B12) using main communication. Furthermore, sub transmission data is transmitted to communication apparatus B12 using sub communication. Also, when no signal notification or recovery notification is notified from the signal detection unit 603, when blocking information is notified from the blocking information notification unit 841, or when a display switching request is notified from the display switching request notification unit 842 The video transmission unit 861 transmits the no-signal notification, the recovery notification, the cutoff information, or the display switching request to the communication device B12 using the main communication or the sub communication. For example, these notifications are transmitted using an RTCP packet APP (Application-defined) packet.

  As described above, the video communication apparatus 800 (communication apparatus A11) according to the present embodiment transmits information indicating the presence / absence of a sub input signal, blocking information, and a display switching request to the communication apparatus B12.

  Subsequently, a functional configuration of the communication device B12 according to the present embodiment will be described.

  FIG. 17 is a block diagram illustrating a functional configuration of the video communication apparatus 900 on the side receiving the sub input signal according to the present embodiment. The video communication device 900 corresponds to the communication device B12, and is a device that performs processing related to transmission / reception of video signals. As shown in the figure, the video communication apparatus 900 includes a communication unit 906 instead of the communication unit 706 and a display data generation unit 907 instead of the display data generation unit 707, as compared with the video communication apparatus 700 of FIG. Is different. In the following, description of the same points as in the second embodiment will be omitted, and different points will be mainly described.

  The communication unit 906 transmits and receives data using two communications, main communication and sub-communication. As illustrated in FIG. 17, the communication unit 906 includes a video transmission unit 761 and a video reception unit 962. Note that the video transmission unit 761 performs the same processing as in the second embodiment, and a description thereof will be omitted here.

  The video receiver 962 receives main transmission data from the video communication device 700 (communication device A11) using main communication. Furthermore, sub transmission data is received from communication apparatus A11 using sub communication. In addition, a no-signal notification, a recovery notification, blocking information, or a display switching request is received, and the received information is output to the display data generation unit 907.

  The display data generation unit 907 performs the same processing as the display data generation unit 707. The display data generation unit 707 of the second embodiment is different from the display data generation unit 707 in that a no-signal notification or the like is received from the video reception unit 962.

  As described above, the video communication apparatus 900 (communication apparatus B12) of the present embodiment receives information (no-signal notification and recovery notification) on the presence / absence of a sub input signal, blocking information, and display switching request from the communication apparatus A11. The display data is generated using the received information. That is, the communication device A11 on the side that transmits the sub-input signal determines at what timing the blocking information is displayed on the display 721 and at what timing the video display is changed, as in the first embodiment.

  Next, the operation of the communication device according to the present embodiment will be described. An image displayed on the display 721 of the communication device B12 will be described with reference to FIG.

  Further, since the process from establishing main communication and sub-communication and starting content sharing is the same as that of the second embodiment (FIG. 14), description thereof is omitted here. Furthermore, since the operation of the video communication apparatus 800 on the side that transmits the sub input signal is the same as that of the first embodiment, the description thereof is omitted here. Therefore, hereinafter, the operation of the video communication apparatus 900 (communication apparatus B12) will be mainly described.

  FIG. 18 is a flowchart showing an operation when the sub input signal is blocked in the communication device B12. Note that a video 211 is displayed on the display 721 when content sharing is started. At this time, when the sub input signal is cut off on the communication device A11 side, the video 221 is displayed on the display 721. At this time, the blocking information is transmitted from the blocking information notification unit 841 of the communication device A via the video transmission unit 861.

  The video reception unit 962 receives the cutoff information and outputs the received cutoff information to the display data generation unit 907 (S501). The display data generation unit 907 generates display data including the received blocking information and the first main video. The generated display data is output to the display 721 via the video output unit 708. At this time, the video 231 in FIG. 8 is displayed on the display 721.

  If the video reception unit 962 receives the restoration notification (Yes in S502), the display data generation unit 907 generates display data including the first main video and the sub video. The generated display data is output to the display 721 via the video output unit 708. At this time, the video 211 in FIG. 8 is displayed on the display 721.

  When the video reception unit 962 receives the display switching request without receiving the recovery notification (No in S502), the display data generation unit 907 sets the parent screen as the main video included in the main input signal, and sets the child screen as the sub screen. The sub video included in the sub input signal is set (S503). Thereby, the video 241 in FIG. 8 is displayed.

  Here, when the video reception unit 962 receives the restoration notification (Yes in S504), the display data generation unit 907 generates display data including the first main video and the sub video. The generated display data is output to the display 721 via the video output unit 708. At this time, the video 211 in FIG. 8 is displayed on the display 721.

  When the video receiving unit 962 receives the content sharing cancellation request without receiving the restoration notification (No in S504), the display data generating unit 907 displays the display data so that only the first main video is displayed. Generate. The generated display data is output to the display 721 via the video output unit 708. At this time, the image 201 in FIG. 8 is displayed on the display 721.

  As described above, the video communication system (electronic conference system) using the video communication apparatus 800 (communication apparatus A11) and the video communication apparatus 900 (communication apparatus B12) of the present embodiment is similar to the second embodiment. Video data is transmitted and received using two communications, main communication and sub communication. At this time, the video communication device 900 on the side of receiving the sub video displays the cutoff information according to the no-signal notification, the cutoff information, the display switching request, and the like transmitted from the video communication device 800 on the side of transmitting the sub video. And switching the display.

  Thereby, even if it is a case where two communication, main communication and sub communication, is used, interruption | blocking information is displayed on both display parts. Therefore, the participant participating in the electronic conference system can easily confirm that the sub input signal is blocked by looking at the blocking information displayed on the display 721.

  Furthermore, as in the first and second embodiments, the video communication apparatuses 800 and 900 according to the present embodiment automatically display the sub-video from the parent screen to the child screen after notifying the cutoff information without any participant operation. Change. Therefore, even if the participant is not familiar with the operation of the present apparatus, the electronic conference can be continued smoothly.

  The video communication apparatus, the video communication method, and the video communication system according to the present invention have been described based on the embodiments. However, the present invention is not limited to these embodiments. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to the said embodiment, and the form constructed | assembled combining the component in a different embodiment is also contained in the scope of the present invention. .

  Further, in the communication device A11 on the side of transmitting the sub video, it is not necessary to display the sub video on the main screen of the display 21 (display unit 41), and it may be displayed on the sub screen. Alternatively, the sub video may not be displayed. That is, the main video may be displayed on the full screen.

  In the second embodiment, the communication apparatus B12 that receives the sub input signal performs the generation of the blocking information, the notification of the display switching request, and the notification of the content sharing cancellation request. In 1 and 3, these processes are performed by the communication device A11 on the side of transmitting the sub input signal. On the other hand, which process may be executed by which apparatus. For example, the communication device A11 may generate blocking information, and the communication device B12 may notify a display switching request and a content sharing release request.

  As described above, the present invention can be realized not only as a video communication apparatus and a video communication method, but also as a program for causing a computer to execute the video communication method of the present embodiment. Moreover, you may implement | achieve as recording media, such as computer-readable CD-ROM which records the said program. Furthermore, it may be realized as information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

  In the present invention, some or all of the components constituting the video communication apparatus may be configured from one system LSI. The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip. Specifically, the system LSI is a computer system including a microprocessor, a ROM, a RAM, and the like. .

  As described above, the video communication apparatus of the present invention has an effect that the cause of the no-signal state of the screen display can be clearly understood, and can be used for an electronic conference system or the like.

1 is a diagram showing a configuration of a communication system at two bases in Embodiment 1. FIG. 1 is a diagram showing an example of a configuration of a communication system at multiple locations in Embodiment 1. FIG. 3 is a diagram illustrating an example of an arrangement of input / output devices included in the communication device in Embodiment 1. FIG. 3 is a block diagram illustrating an example of a hardware configuration of a communication device according to Embodiment 1. FIG. 3 is a block diagram illustrating a functional configuration of a video communication device on a side that transmits a sub input signal in Embodiment 1. FIG. 4 is a flowchart illustrating an operation for establishing main communication and an operation for starting content sharing in the communication device according to the first embodiment. 4 is a flowchart showing an operation when a sub input signal of the communication device in Embodiment 1 is interrupted. 6 is a diagram illustrating an example of image transition in the display unit of the communication device on the side of receiving a sub input signal in Embodiment 1. FIG. 6 is a diagram showing an example of image transition in the display unit of the communication device on the side of transmitting a sub input signal in Embodiment 1. FIG. 6 is a diagram illustrating an example of a screen including a cancellation notification in the display unit of the communication device according to Embodiment 1. FIG. 6 is a diagram showing an example of different image transitions displayed on the display unit of the communication device on the side of receiving a sub input signal in Embodiment 1. FIG. 10 is a block diagram illustrating a functional configuration of a video communication apparatus on a side that transmits a sub input signal in Embodiment 2. FIG. 6 is a block diagram illustrating a functional configuration of a video communication device on a side that receives a sub input signal in Embodiment 2. FIG. 10 is a flowchart showing an operation for establishing main communication and sub-communication and an operation for starting content sharing in the communication apparatus according to the second embodiment. 10 is a flowchart showing an operation when the sub input signal of the communication device on the side receiving the sub input signal in the second embodiment is blocked. FIG. 10 is a block diagram illustrating a functional configuration of a video communication apparatus on a side that transmits a sub input signal according to a third embodiment. 10 is a block diagram illustrating a functional configuration of a video communication apparatus on a side that receives a sub input signal in Embodiment 3. FIG. 10 is a flowchart showing an operation when a sub input signal of a communication device on the side of receiving a sub input signal in Embodiment 3 is cut off.

Explanation of symbols

10 Network 11 Communication device A
12 Communication device B
13 Notebook PC
14 Camera 15 PDA
16 Mobile phone 17 Server 18 Internet 19 Desktop PC
20 Main body 21, 721 Display 22, 722 Main camera 23 Microphone 24a, 24b, 24c Speaker 25 Desk 26 Remote control 27 Sub camera 28 Notebook PC
30 control unit 31 audio encoding unit 32 audio decoding unit 33 image encoding unit 34 image decoding unit 35 power supply circuit 36 timer circuit 37 audio processing unit 38 audio output unit 39 audio input unit 40 display processing unit 41 display unit 42 image Processing unit 43 Image input unit 44 Operation input control unit 45 Operation input unit 46 Communication control unit 47 Transmission / reception circuit 50 CPU
51 ROM
52 RAM
61 Signal confirmation unit 62, 104, 604, 704, 804 Notification unit 100, 600, 700, 800, 900 Video communication device 101, 701 Main video input unit 102 Sub video input unit 103, 603 Signal detection unit 105, 605, 705 Transmission data generation unit 106, 606, 706, 806, 906 Communication unit 107, 707, 907 Display data generation unit 108, 708 Video output unit 141, 641, 741, 841 Blocking information notification unit 142, 642, 742, 842 Display switching Request notification unit 161,661,761,861 Video transmission unit 162,662,762,962 Video reception unit 201,211,221,231,241,301,311,321,331,341,441,501,511,521 531 video 212, 222, 232, 242, 312 322, 332 and 342 parent screen 213,223,233,243,313,323,333,343 child screen 234,334,534 blocking information 444 release information

Claims (16)

A video communication device that communicates with other video communication devices,
First input means for acquiring a first video signal indicating a first video;
Second input means for acquiring a second video signal indicating the second video;
Transmission data generating means for generating first transmission data including at least the second video out of the first video and the second video;
Communication means for transmitting the first transmission data generated by the transmission data generating means to the other video communication apparatus and receiving reception data including a third video from the other video communication apparatus;
Display data generating means for generating display data including the third video image using received data received by the communication means;
Output means for outputting the display data generated by the display data generating means to the first display device;
Signal detection means for detecting whether or not the second video signal is acquired by the second input means;
When the signal detecting means detects that the second video signal is not acquired, it generates blocking information indicating that the second video signal is blocked, and generates the generated blocking information as the transmission data. A notification means for notifying the display data generation means and the display data generation means,
The transmission data generating means generates second transmission data including the blocking information when the blocking information is notified,
The communication unit transmits the second transmission data generated by the transmission data generation unit to the other video communication device;
The display data generation unit generates the display data including the block information and the third video when the block information is notified. The transmission data generating means generates the first transmission data so that the second video is displayed on the second display device by the other video communication device,
The notification means further notifies the transmission data generation means of a display switching request indicating that the display is to be switched,
The video communication device according to claim 1, wherein the transmission data generation unit generates the second transmission data so that the first video is displayed on the second display device when the display switching request is notified. The transmission data generating means displays the second video in a predetermined first area of the second display device, and displays the first video in a second area having a size equal to or smaller than the first area. 3. The video communication according to claim 2, wherein when the first transmission data is generated and the display switching request is notified, the second transmission data is generated so that the first video is displayed in the first area. apparatus. The said notification means produces | generates the said interruption | blocking information, when it detects that the said 2nd video signal is not acquired continuously for the 1st predetermined period by the said signal detection means. Video communication device. The video according to any one of claims 2 to 4, wherein the notification unit notifies the transmission data generation unit of the display switching request after a predetermined second period has elapsed since the generation of the blocking information. Communication device. The notification means further notifies the transmission data generation means of a release request indicating that only the first video is displayed after a predetermined third period has elapsed since the notification of the display switching request.
The video according to any one of claims 2 to 5, wherein, when the release request is notified, the transmission data generation unit generates the second transmission data so that the first video is displayed on a full screen. Communication device. When the transmission data generation unit detects that the second video signal has been acquired again after the signal detection unit detects that the second video signal has not been acquired, The video communication device according to claim 2, wherein the first transmission data including video is generated. 4. The video communication apparatus according to claim 2, wherein the notifying unit receives a stop command indicating that the second video signal is to be stopped from a user and simultaneously outputs the display switching request to the transmission data generating unit. The transmission data generation unit generates the second transmission data so that the blocking information is displayed as character information in a predetermined third area of the second display device. The video communication apparatus described. The video communication device according to any one of claims 1 to 8, wherein the transmission data generation unit further generates the second transmission data so that the blocking information is output as a sound. The video communication device further includes:
The video communication device according to claim 1, further comprising an information display unit that displays the blocking information among the display data. A video communication system comprising a first video communication device and a second video communication device,
The first video communication device
First input means for acquiring a first video signal indicating a first video;
Second input means for acquiring a second video signal indicating the second video;
First transmission data generating means for generating first transmission data including the first video and second transmission data including the second video;
A first communication and a second communication are established with the second video communication device, the first transmission data is transmitted using the established first communication, and a third video including a third video is transmitted. First communication means for receiving the received data and further transmitting the second transmission data using the established second communication;
First display data generating means for generating first display data including the third video image using third received data received by the first communication means;
First output means for outputting the first display data generated by the first display data generating means to the first display device;
Whether or not the second video signal is acquired by the second input means is detected, and when the second video signal is not acquired, a no-signal notification indicating that the second video signal is not acquired A signal detection means for notifying one communication means;
When the signal detecting means detects that the second video signal is not acquired, it generates blocking information indicating that the second video signal is blocked, and displays the generated blocking information in the first display. First notification means for notifying the data generation means,
The first communication means transmits the no-signal notification using the first communication or the second communication when the no-signal notification is notified,
The first display data generating means generates the first display data including the blocking information and the third video when the blocking information is notified,
The second video communication device is:
Third input means for acquiring a third video signal indicating the third video;
Second transmission data generating means for generating the third reception data including the third video;
Second communication that receives the first transmission data using the first communication, transmits the third reception data, and further receives the second transmission data using the second communication. Means,
Second display data generating means for generating second display data including at least one of the first video and the second video;
Second output means for outputting the second display data generated by the second display data generating means to a second display device;
Second notification means for generating the blocking information and notifying the generated display information to the second display data generating means when the second communication means receives the no-signal notification from the first video communication device; With
The said 2nd display data production | generation means produces | generates the display data containing the said interruption | blocking information, when the said interruption | blocking information is notified. A video communication system comprising a first video communication device and a second video communication device,
The first video communication device
First input means for acquiring a first video signal indicating a first video;
Second input means for acquiring a second video signal indicating the second video;
First transmission data generating means for generating first transmission data including the first video and second transmission data including the second video;
A first communication and a second communication are established with the second video communication device, the first transmission data is transmitted using the established first communication, and a third video including a third video is transmitted. First communication means for receiving the received data and further transmitting the second transmission data using the established second communication;
First display data generating means for generating first display data including the third video image using third received data received by the first communication means;
First output means for outputting the first display data generated by the first display data generating means to the first display device;
Signal detection means for detecting whether or not the second video signal is acquired by the second input means;
When the signal detecting means detects that the second video signal is not acquired, it generates blocking information indicating that the second video signal is blocked, and displays the generated blocking information in the first display. First notification means for notifying the data generation means and the first communication means;
When the blocking information is notified, the first communication means transmits the blocking information using the first communication or the second communication,
The first display data generating means generates the first display data including the blocking information and the third video when the blocking information is notified,
The second video communication device is:
Third input means for acquiring a third video signal indicating the third video;
Second transmission data generating means for generating the third reception data including the third video;
Second communication that receives the first transmission data using the first communication, transmits the third reception data, and further receives the second transmission data using the second communication. Means,
Second display data generating means for generating second display data including at least one of the first video and the second video;
Second output means for outputting the second display data generated by the second display data generating means to a second display device;
The said 2nd display data production | generation means produces | generates the display data containing the said interruption | blocking information, when the said interruption | blocking information is notified The video communication system. A video communication method for communicating with other video communication devices,
A first input step of obtaining a first video signal indicating the first video;
A second input step of obtaining a second video signal indicating the second video;
A transmission data generation step of generating first transmission data including at least the second video out of the first video and the second video;
A communication step of transmitting the first transmission data generated in the transmission data generation step to the other video communication device and receiving reception data including a third video from the other video communication device;
A display data generation step of generating display data including the third video using the reception data received in the communication step;
An output step of outputting the display data generated in the display data generation step to the first display device;
A signal detection step of detecting whether or not the second video signal is acquired in the second input step;
A notification step of generating blocking information indicating that the second video signal has been blocked and notifying the generated blocking information when it is detected that the second video signal is not acquired in the signal detection step; Including
When the blocking information is notified in the transmission data generation step, the second transmission data including the blocking information is generated,
In the communication step, the second transmission data generated in the transmission data generation step is transmitted to the other video communication device,
In the display data generation step, when the blocking information is notified, the display data including the blocking information and the third video is generated. An integrated circuit that communicates with other video communication devices,
First input means for acquiring a first video signal indicating a first video;
Second input means for acquiring a second video signal indicating the second video;
Transmission data generating means for generating first transmission data including at least the second video out of the first video and the second video;
Communication means for transmitting the first transmission data generated by the transmission data generating means to the other video communication apparatus and receiving reception data including a third video from the other video communication apparatus;
Display data generating means for generating display data including the third video image using received data received by the communication means;
Output means for outputting the display data generated by the display data generating means to the first display device;
Signal detection means for detecting whether or not the second video signal is acquired by the second input means;
When the signal detecting means detects that the second video signal is not acquired, it generates blocking information indicating that the second video signal is blocked, and uses the generated blocking information as the transmission data generating means. And notifying means for notifying the display data generating means,
The transmission data generating means generates second transmission data including the blocking information when the blocking information is notified,
The communication unit transmits the second transmission data generated by the transmission data generation unit to the other video communication device;
The display data generation means generates the display data including the block information and the third video when the block information is notified. A program for causing a computer to execute a video communication method for communicating with other video communication devices,
A first input step of obtaining a first video signal indicating the first video;
A second input step of obtaining a second video signal indicating the second video;
A transmission data generation step of generating first transmission data including at least the second video out of the first video and the second video;
A communication step of transmitting the first transmission data generated in the transmission data generation step to the other video communication device and receiving reception data including a third video from the other video communication device;
A display data generation step of generating display data including the third video using the reception data received in the communication step;
An output step of outputting the display data generated in the display data generation step to the first display device;
A signal detection step of detecting whether or not the second video signal is acquired in the second input step;
A notification step of generating blocking information indicating that the second video signal has been blocked and notifying the generated blocking information when it is detected that the second video signal is not acquired in the signal detection step; Including
When the blocking information is notified in the transmission data generation step, the second transmission data including the blocking information is generated,
In the communication step, the second transmission data generated in the transmission data generation step is transmitted to the other video communication device,
In the display data generation step, when the blocking information is notified, the display data including the blocking information and the third video is generated.

Images