KR100346735B1 - Three-dimensional video conferencing system in consideration of eye contact among participants - Google Patents

Abstract

PURPOSE: A three-dimensional video conferencing system in consideration of eye contact among participants is provided to allow participants to keep eye contact in a video conferencing system in which at least two people participate. CONSTITUTION: A three-dimensional video conferencing system in consideration of eye contact among participants includes an image acquisition unit(101), an image matching unit(102), and a transmitter. The image acquisition unit acquires an image of at least one conference participant located at a fixed position using three cameras(100). The image matching unit selects a representative image from images acquired by the image acquisition unit and generates image models with respect to the acquired images. The transmitter includes a communication controller(103) for transmitting a representative image and an image model output from the image matching unit to another video conferencing system.

Description

3D video conferencing system considering eye contact between participants

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional video conference system, and more particularly, to a three-dimensional video conference system in consideration of eye-contact between a large number of participants.

As a prior art, US Patent No. No. 5,359,362 assumes that there is only one participant in front of one monitor at a site, and that two cameras are placed on the top / bottom or left / right of the monitor and are calibrated by two cameras An image is obtained and stereo matching is performed on these two images to calculate a disparity map. With the calculated mismatch map and one image, the rotation process creates a virtual image as seen from the center of the monitor and maintains i-Contec by displaying it on the monitor of the other site. However, this method can be effectively used in a videophone by limiting the number of participants to one person, but has a disadvantage in that it is difficult to apply in a video conference system in which a large number of people must attend.

Meanwhile, Klaus Hopf (see Advanced video communications with stereoscopy and individual perspectives, 1994) assumes two participants at a site, puts a camera in front of each person, acquires images from two viewpoints, and then It presents a way to maintain the I-contact between each participant by selectively displaying the two participants. However, this method has a disadvantage in that it is difficult to configure the system because it is necessary to install a camera corresponding to twice the number of participants when applied to a video conferencing system having a large number of participants.

Accordingly, an object of the present invention is to provide a three-dimensional video conferencing system for maintaining the i-contact between participants in a video conferencing system in which at least two people can attend in order to solve the above problems.

In order to achieve the above object, the three-dimensional video conference system according to the present invention

An image input unit for acquiring images of at least one or more conference participants in a fixed position using three cameras,

An image matching unit for selecting a representative image among the images acquired by the image input unit and generating an image model of the acquired images; And

And a transmitting means having a communication control unit for transmitting the representative image and the image model output from the image matching unit to the other video conferencing system.

In order to achieve the above object, the three-dimensional video conference system according to the present invention

Communication control unit for receiving the representative image and the image model sent from the transmitting side;

A 3D image reconstruction unit for reconstructing 3D images to be displayed to each conference participant using the representative image and the image model received through the communication control unit;

A display control unit which controls to accurately display the restored 3D image with reference to the number and location of each conference participant; And

And a receiving unit having a display unit for displaying an image capable of maintaining an i-contact to each conference participant under the control of the display control unit.

In order to achieve the above object, the three-dimensional video conference system according to the present invention

An image input unit for acquiring images of at least one or more conference participants in a fixed position using three cameras;

An image matching unit for selecting a representative image among the images acquired by the image input unit and generating an image model of the acquired images;

A communication controller for transmitting and receiving one representative image and an image model outputted from the image matching unit to another video conference system;

A 3D image reconstruction unit for reconstructing 3D images to be displayed to each conference participant using the representative image and the image model received through the communication control unit;

A display control unit which controls to accurately display the restored 3D image with reference to the number and location of each conference participant; And

And a display unit configured to display an image capable of maintaining the i-Contec to each conference participant under the control of the display controller.

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is an overall configuration diagram of a three-dimensional video conference system according to the present invention, three cameras 100, the video input unit 101, the video input unit 101 to obtain the image of the conference participants from the three cameras 100 Image matching unit 102 for generating a representative image and an image model with respect to the images obtained in the), communication control unit 103 for transmitting and receiving the representative image and the image model output from the image matching unit 102 to the other video conferencing system ), A 3D image restoration unit 104 for restoring 3D images to be displayed to each meeting participant using the representative image and the image model received through the communication controller 103, and the number and position of each meeting participant. For reference, the display control unit 105 controls to accurately display the restored 3D image and maintains the i-contact for each conference participant under the control of the display control unit 105. It is composed of a display unit 106 for displaying an image with.

FIG. 2 is a detailed block diagram of the image matching unit 102 shown in FIG. 1, and includes a spatiotemporal image grouping unit 201, a matching unit 202, a depth map generator 203, and a surface model forming unit 204. It consists of.

3 is a detailed block diagram of the three-dimensional image reconstructing unit 104 shown in FIG. 1, and is composed of a perspective projection unit 301.

Next, with reference to FIGS. 1 to 3 will be described the operation and effect of the present invention.

The image input unit 101 obtains the images of the participants from the three cameras 100 installed on the screen of the conference hall. Here, the three cameras 100 are arranged in the shape of a triangle one above the center of the screen and one each on the left and right of the screen. At this time, the cameras 100 are arranged such that the optical axes are parallel to each other. This has the advantage that the epipolar lines of the left and right cameras are parallel to the scan line. In addition, a system using three cameras has an advantage of reducing occlusion problems and correspondence errors of an image sector compared to a system using two cameras.

The image matching unit 102 determines a match of the images output from the image input unit 101 by a feature-based method, and generates a depth map necessary for generating images corresponding to the viewpoint. Create a surface model. The image matching unit 102 will be described in more detail with reference to FIG. 2 as follows.

In FIG. 2, the temporal / spatial image grouping unit 201 analyzes an image output from the image input unit 101 in consideration of a spatial axis and a time axis at the same time. The image analyzed by the spatiotemporal image grouping unit 201 calculates information necessary to obtain a 3D image by extracting the correlation of the images from the matching unit 202. If the pixels in the top image, right image and left image are Pt, Pr, and Pl, respectively, the coincidence problem between these three pixels is Liu's three eyes which extends the binocular comparison method proposed by Ballard and Horaud as follows. This can be solved using a (trinocular) matching method.

correspndence (Pt, Pr, Pl): = correspndence (Pt, Pr)

and correspndence (Pt, Pl)

and correspndence (Pr, Pl)

When calculating the inconsistencies of each point in this way, there are points where those values are not determined. For these points, the discrepancies can be estimated by interpolating using the mismatches of the neighboring points.

The depth map generator 203 generates a depth map using the estimated inconsistencies, and the surface model forming unit 204 uses the depth map generated by the depth map generator 203 to generate a three-dimensional structure of an image. Express. First, the surface model forming unit 203 makes a patch of triangles or squares using a given depth map. Using the values of these patches, a model is created that contains three-dimensional information about the surface of the image to be displayed. Such model information is transmitted through the communication control unit 103 together with the representative image of any of the left and right images and applied to the 3D image reconstructing unit 104 on the receiving side.

At this time, the communication control unit 103 is a device embedded in the video conferencing system, and is responsible for transmitting the representative image and the image model generated at the transmitting side to the receiving side, or transferring information from the transmitting side to the receiving side. The communication control unit 103 may select various transmission methods, for example, a general public network, an integrated service digital network (ISDN), or a high speed transmission network, depending on the situation in which the video conference system is configured. That is, the communication controller 103 makes data in a data transmission method suitable for this in consideration of various transmission methods and characteristics of a video conference system, and performs a function of exchanging these data.

In the 3D image reconstructing unit 104, as shown in FIG. 3, the perspective projection unit 301 receives the viewpoint as an input and has a representative image to properly map the surface model to maintain the participant's eye contact. To create an image. At this time, as many images as the number of conference participants on the receiving side of the image in one site are generated and transmitted to the display unit 106. The generated image is generated by gradually changing three-dimensional information about the surface of the image and one representative image including the power source of the meeting place of the transmitter. At this time, the viewpoint is made by considering the location of the conference participants according to the video conference system. In other words, the viewpoint is configured appropriately in consideration of the direction of the eyes of the speaker and the position of the other person, or by arbitrarily selecting a position to view the image in the direction desired by the listener. It also determines the viewpoint. Therefore, this viewpoint is used as the most important element in forming the image to be output from the three-dimensional image restoration unit 104.

The display controller 105 selects an image and transmits the image to the display unit 106 in order to properly output the image generated by the 3D image reconstructor 104 according to the location of the conference participant. In other words, the accurate image is displayed by referring to the number of people present and their location.

The display unit 106 is composed of a liquid crystal display (LCD) and a lenticular screen, using a multiview display technology to display an image that can maintain eye contact for each participant. Participants at the location are displayed a predetermined image.

As described above, according to the three-dimensional video conference system according to the present invention, there is an advantage that can maintain the eye contact between the participants in the video conference system that at least two people can attend.

1 is an overall configuration diagram of a three-dimensional video conference system according to the present invention.

FIG. 2 is a detailed block diagram of the image matching unit shown in FIG.

3 is a detailed block diagram of the tertiary image reconstruction unit shown in FIG.

* Explanation of symbols for main parts of the drawings

100 ... Camera 101 ... Video Input

102 ... image matching unit 103 ... communication control unit

104 ... 3D Image Restoration Unit 105 ... Display Control Unit

Claims (12)

An image input unit for acquiring images of at least one or more conference participants in a fixed position using three cameras; An image matching unit for selecting a representative image among the images acquired by the image input unit and generating an image model of the obtained images; And 3D video conferencing system in consideration of i-contact between participants, characterized in that it comprises a transmitting means having a communication control unit for transmitting one representative image and an image model outputted from the image matching unit to the other video conferencing system. . The 3D video conferencing system of claim 1, wherein the three cameras are arranged in a triangular shape in which the optical axes are parallel to each other and symmetrically on the left and right sides of the screen. The 3D video conference system of claim 1, wherein the image matching unit solves a matching problem using a three-eye matching method using three images. The method of claim 1, wherein the image matching unit determines inconsistencies of points for which the inconsistencies are not determined by interpolation using inconsistencies of neighboring points with respect to points for which inconsistencies are not determined. 3D video conferencing system considering i-contact between participants. The 3D video conference system of claim 1, wherein the image matching unit creates a model including 3D information on the surface of the image by using a patch. 6. The three-dimensional video conference system according to claim 5, wherein an eye contact between participants is used by forming a triangular patch to form the patch. 6. The 3D videoconferencing system of claim 5, wherein an eye contact between participants is used to form a patch. Communication control unit for receiving the representative image and the image model sent from the transmitting side; A 3D image reconstruction unit for reconstructing 3D images to be displayed to each conference participant using the representative image and the image model received through the communication control unit; A display control unit which controls to accurately display the restored 3D image with reference to the number and location of each conference participant; And And a receiving unit having a display unit for displaying an image capable of maintaining an i-contact to each conference participant under the control of the display control unit. The 3D video conferencing system of claim 8, wherein the 3D image reconstructor restores 3D images to be displayed to each conference participant using a viewpoint. The 3D video conferencing system of claim 9, wherein the viewpoint is automatically generated in consideration of the position of the speaking person and the position of the other party. 10. The system of claim 9, wherein the viewpoint is selected according to the intention of the conference participant. An image input unit for acquiring images of at least one or more conference participants in a fixed position using three cameras; An image matching unit for selecting a representative image among the images acquired by the image input unit and generating an image model of the acquired images; A communication controller for transmitting and receiving one representative image and an image model outputted from the image matching unit to another video conference system; A 3D image reconstruction unit for reconstructing 3D images to be displayed to each conference participant using the representative image and the image model received through the communication control unit; A display control unit which controls to accurately display the restored 3D image with reference to the number and location of each conference participant; And And a display unit configured to display an image capable of maintaining an i-contact to each conference participant under the control of the display control unit.