JP2004178036A - Device for presenting virtual space accompanied by remote person's picture - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To dynamically select a picture by matching pictures of a transmission user photographed from two or more angles with positional relationship between the transmission user and a reception users in a virtual space, to adjust the picture size and the frame rate, and then to transfer the picture by using a network, so that picture synthesis is performed while suppressing a load imposed on the network. <P>SOLUTION: A photographing system 101 which photographs with a plurality of cameras and a display system 102 which displays a picture synthesized with the virtual space being a background picture are combined via the network 103. An avatar character can be viewed whose angle and resolution are varied in conformity with a position thereof in the virtual space. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a photographing / display device that performs communication and cooperative work by transferring a moving image in real time.
[0002]
[Prior art]
2. Description of the Related Art A device that captures an image of a user, transfers the image in real time, and views the image, thereby realizing communication as if the user is present at a remote place has been put to practical use. The most typical one is called a video conferencing system, which captures the image of a sender in a remote place with a camera and transmits the image together with voice to a display system on the receiving side in real time. With the accompanying image, communication with more information and a larger amount of information can be achieved than in a communication system using only voice such as a telephone.
[0003]
Also, by separating the part of the image where the person is shown from the part where the background is shown, instead of the actual background image, the image of the virtual space constructed on the computer and the person image are synthesized, Research has been conducted in various places to add further added value to a remote video transfer system. These studies are called video avatars and there are many examples.
[0004]
However, if only one camera is used to capture the user, only one direction of video can be obtained. Cannot be obtained. For this reason, there are many cases where the image of the user (transmitting side) always becomes a front-facing image on the receiving system. For example, Patent Document 1 relates to an improved example of a video avatar in consideration of such a problem. By capturing images of the face from multiple directions and converting them into three-dimensional information, or transmitting all images from all directions, and combining them with a three-dimensional shape model representing the body, arbitrary images in the virtual space can be obtained. Video from the position of In addition, since the amount of information to be transferred in a video is large and the communication network is squeezed, the video transfer amount is reduced by narrowing down the video to the face portion. However, for relatively simple and constant structures such as faces, it is easy to reproduce 3D information.However, 3D information of complex shapes, such as the upper body including hands, is captured only from the information of the image to be captured. Difficult to reproduce. In addition, a simple 3D model obtained by pasting an image of a user's image onto a 3D model is simple in shape due to the problem of calculation accuracy. An unnaturally shaped part is often formed, which may give an unnatural impression.
[0005]
[Patent Document 1]
Patent application No. 10-257702
[0006]
[Problems to be solved by the invention]
The image information required by the display device in the virtual space differs depending on the positional relationship between the video receiver and the video sender in the virtual space. For example, when the receiver and the sender are at a distance from each other in the virtual space, a small image of the whole body of the sender is displayed. Conversely, if the receiver and the sender are very close, only the upper body of the sender, especially the face, is displayed large. In addition, a natural impression can be given if the viewed image is different depending on the angle of the receiver with respect to the front of the sender. For example, if the recipient is on the side of the sender, the sender's view from the recipient should be only the side view. However, when trying to reproduce these images in a state where the positional relationship between the two cannot be clearly grasped, these image information comes from a fixed angle, and the image quality is always constant. Will be.
[0007]
Further, since the transfer capacity of the line for transferring the video is limited, it is difficult to transfer the video that simultaneously satisfies many demands. For example, it is difficult to send a moving image that satisfies both smooth motion and a large video image. In order to achieve smooth movement, the video image is small, and if the video image is large, the video image has a low frame rate. In addition, transferring all of the images viewed from each direction greatly reduces the transfer capacity of the line.
[0008]
Means for Inventing the Object
In the present invention, the receiving and displaying system determines necessary video conditions based on the positional relationship between the sender and the receiver in the virtual space. This information is transferred to the image capturing system, which converts the image at the selected angle from among the images captured by the multiple cameras into an image of an appropriate size in terms of time and space by means of thinning and trimming. Modify and transfer to display system. As a result, required video can be transmitted with a small transfer capacity.
[0009]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 illustrates an embodiment of the present invention. The present invention is broadly constructed by an imaging system 101, a display system 102, and a communication network facility 103 connecting them.
[0010]
FIG. 2 is a detailed configuration example of the photographing system 101. A plurality of camera devices 201 to 205 are installed so as to surround the transmission user 200. A background cloth 207 of the same color is installed around the user 200 for chroma key processing. It is also assumed that the input device 209 operated by the transmission user 200 and the lower body of the transmission user 200 are wrapped with the same color cloth 209 so as not to be obtained as video information. The video information of these cameras is transferred to the information processing device 210. Each camera 201 to 205 is assigned a unique ID number, and the information processing device 210 can select a video to be captured based on the ID number. The information processing apparatus has a function of receiving images from these cameras, a function of transmitting and receiving information on the network 103, and a function of calculating and processing the information.
[0011]
FIG. 3 is a block diagram illustrating a configuration example of the information processing device 210. A CPU (Central Processing Unit) 301 executes various processes according to a program recorded in a main storage device 302. The main storage device 302 and the external storage device 304 store programs and data necessary for executing the control processing. It is assumed that a hard disk drive or other existing large-capacity medium is used for the external storage device 304. The input / output interface unit 303 is connected to a data transfer mechanism 507 necessary for exchanging input / output data with the camera devices 201 to 205 to and from the network. The output device 305 is an output device for transmitting the current processing status to the transmission user. The input device 209 is an input device to which the transmitting user can transmit some commands. In this embodiment, the input device 209 is an input device that can be operated by a foot.
[0012]
FIG. 4 is a configuration example of a display system part. The video display device 401 is placed in front of the receiving user 400. The video output from the information processing device 402 is displayed on this device. The receiving user 400 can give a command to the information processing device 402 via the input device 403. The information processing device 402 has a function of creating various images using three-dimensional shape data, two-dimensional video data, and two-dimensional moving image data, a function of transmitting and receiving network information, and a calculation process for the information. It has a function to perform.
[0013]
FIG. 5 is a block diagram illustrating a configuration example of the information processing device 402. A CPU (Central Processing Unit) 501 executes various processes according to programs recorded in a main storage device 502. The main storage device 502 and the external storage device 504 store programs and data necessary for executing the control processing. It is assumed that a hard disk drive or another existing large-capacity medium is used for the external storage device 504. It is assumed that the input / output interface unit 503 has a data transfer device 507 necessary for exchanging input / output data with the network 103 and a processing capability for outputting a video to the video display device 401.
[0014]
FIG. 6 shows a schematic diagram of virtual space information constituted by the information processing device 402 and the internal data of the information processing device 210.
[0015]
A virtual space 601 as a background is configured by three-dimensional shape data of background objects 602 and 603 held by the display system. Characters 610 and 620 representing the actions of the transmitting user 200 and the receiving user 400 are arranged in this virtual space.
[0016]
A position 621 in the virtual space where the character of the receiving user exists and a direction 622 in which the character faces are defined, and are controlled according to the operation of the receiving user.
[0017]
Further, a character 603 imitating the transmitting user is also configured in the virtual space, and is arranged in the background space 601. The position 611 in the virtual space where the character of the transmitting user exists and the direction 612 in which the character faces are also controlled according to the operation of the transmitting user.
[0018]
FIG. 7 shows a method of forming the character 610 of the transmission user.
[0019]
The foot 703 of the character is constituted by three-dimensional shape data. The shape data has animation information, and can receive script data to deform the shape. FIG. 8 is a configuration example of animation data representing an action. Numerical values described in 801 are ID data representing operation patterns. In the script data, this ID data is transferred. In order to execute the script data, a search is made from a series of held animation data for a data in which the animation of the data 801 matches the ID number of the script. The loading of the animation data is performed by calling each joint data of the polygon model. The number 802 in the animation data 800 describes the number of basic frames. Reference numeral 803 describes time interval data between frames. Reference numerals 804 to 804 describe rotation information data of each joint. The animation operation is performed by calculating the time from the animation start time to the display time, reading the rotation information data of the joint, and deforming the polygon.
[0020]
The upper body portion 702 of the character is indicated by displaying frame information of a moving image on a polygon. This moving image is obtained by removing a chroma key color portion from a portion 712 of a moving image obtained by capturing an image of the sender 200. Since the lower body part of the sender is surrounded by the cover 208, it is not captured in the image. The polygon on which the moving image is displayed rotates from the position of the transmission character in the virtual space to the surface of the receiver in the virtual space. The polygon for presenting the front of the image to the user by changing the direction in this way is hereinafter referred to as a billboard.
[0021]
The face 701 of the character is composed of a billboard, like the upper body. The moving image displayed on the billboard is obtained by removing a chroma key portion from a portion 711 of a moving image obtained by capturing the image of the sender 200. The image resolution when drawn on the billboard differs between the face 701 of the character and the upper body 702 of the character, and the image 701 of the face generally has a higher pixel density.
[0022]
Hereinafter, a character in the virtual space that reflects the action of the user such as 610 is referred to as an avatar.
In the virtual space 600 constructed as the internal data of the display system 210, the avatar of the sender exists at the position 611 (S_p). The direction 612 in which the avatar 610 of the sender is facing is defined as S_d. In addition, a virtual viewpoint position of the receiver 400 exists at another position 621 (O_p) in the virtual space 600. It is assumed that the direction 622 to which the receiver faces in the virtual space is O_d. These values are represented by vector values.
[0023]
The general-purpose network 103 has a function of transmitting data that the display system transfers to each other. In addition, each system has a function of obtaining information on a transfer capacity available for transfer in the current network by inputting a specific signal to the network.
[0024]
FIG. 9 shows a data structure transferred from the display system to the photographing system in this embodiment. FIG. 10 shows a data structure transferred from the imaging system to the display system. The meaning of the internal data of these structures will be described along with the explanation of the processing operations described below.
[0025]
In the following, the flow of processing in which the above devices operate will be described in order. FIGS. 11 and 12 are block diagrams showing a series of operation order.
[0026]
FIG. 11 describes processing for determining setting information of data to be transferred from the imaging system. FIG. 12 describes processing for transferring data from the imaging system to the display system. In both processes, the information processing devices 210 and 402 are called and executed at regular intervals by using a timer function held in the CPUs 301 and 501.
[0027]
The series of processes in FIG. 11 is hereinafter referred to as an information acquisition thread. This process is performed at a frequency of once to several times per second according to a set value predetermined in the program. The information acquisition thread has a program placed in the information processing 402 and its work content described in the information processing device 210, and the work is started by a timer function of the information processing 402.
[0028]
The series of processes in FIG. 12 is hereinafter referred to as an image transfer thread. This process is called at a rate of several seconds to several tens of times, and the frequency of the call is set by the information acquisition thread. The image transfer thread has a program placed in the information processing 210 and its work content described in the information processing device 402, and the work is started by a timer function of the information processing 210.
[0029]
First, a series of operations of the information acquisition thread will be described with reference to FIG.
The relative position of the avatar 610 of the photographing user 200 with respect to the position of the avatar 620 of the display user 400 is obtained by subtracting the position vector 621 (O_p) from the position vector 611 (S_p) (step 1101).
The angle difference between this direction and the direction 622 to which the avatar 620 of the receiver faces is calculated based on the following formula (Equation 1) (step 1102).
[0030]
(Equation 1)
cos θ = (S_p−O_p) · O_d / (| S_p−O_p || O_d |)
This value is compared with the angle of view W_h of the video display device 402 to determine whether the avatar 610 is within the range of the display environment (step 1103).
If cos θ <= cos (W_h), the process proceeds to step 1104.
If cos θ <cosW_h, there is no need to transfer data because it is outside the display range. In this case, 0 is written in the distance information of the data structure (step 1106), and the process proceeds to step 1107.
In step 1104, the direction φ in which the sender's avatar is facing the receiver is calculated. A vector obtained by rotating (O_p−S_p) around the upward vector by −S_d is defined as O_p1. This vector represents the relative position of the receiver as viewed from the sender's avatar. A value obtained by converting O_p1 into a musical coordinate expression is defined as rφ. φ is a unit length vector expressing the direction of O_p1 viewed from the origin, and r is a value representing the distance from the origin to O_p1 (that is, the distance from O_p to S_p).
[0031]
Write r in the transmission information 1001 of the data structure 1000, and write φ in 1002, and proceed to the process. (Step 1105)
The display system transmits the data structure 1000 to the imaging system 101. (Step 1107)
The imaging system receives the data structure 1000 from the display system. (Step 1108) The angle information 1002 of the data structure is read, and the camera located closest to the corresponding angle is selected (Step 1109). The relationship between the camera ID and the angle is described in FIG. 14 described later.
In step 1110, a required resolution is calculated from the distance information 1001 of the data structure. The required resolution can be obtained by referring to a table defined on the imaging system side. A table 1300 in FIG. 13 expresses a configuration example of this table.
[0032]
In this table, various data necessary for transferring a face image are defined in association with distance values.
[0033]
The distance information (column 1301) described in the table is read in order, and when the distance closest to the value of the distance read from the data 1001 of the data structure 1000 is found, the information associated with the distance is read. Read as resolution information for the face image. This data is composed of two integer values D1 and D2 described in columns 1302 and 1303, and is used in a later-described step as data for changing the resolution of an image. Further, the data size of the face image required for one frame is read from the column 1304. These values are recorded on the main memory, and the process proceeds to step 1111.
[0034]
Further, a similar table is separately prepared for obtaining resolution information of the upper body, and these data are read in the same manner.
[0035]
In step 1111, the transmission frame rate is calculated from the resolution information obtained in the above steps. It sends instructions to the network 103 and receives the amount of data per second available on the current line. The transferable data amount per second is divided into the transfer data transfer capacity for the face image video and the transfer data transfer capacity for the upper body image. For this ratio value, a positive decimal value which is given as a system fixed value and sums to 1 is used. The ratio can be changed by the sender 200 inputting a numerical value with the input device 305.
[0036]
A value obtained by dividing the data transfer capacity of the face image by the data size required for the process 1110 frames is defined as the face image frame rate. However, when the frame rate exceeds the frame rate that can be processed by the transmission mechanism, the upper limit is set to the frame rate of the face image. When 0 is described in the distance information 1001, 0 is set in the frame rate of the face image.
[0037]
Similarly, the frame rate of the upper body image is obtained by dividing the data transfer capacity of the upper body image by the data size required for the upper body image of one frame. However, if the frame rate exceeds the frame rate that can be processed by the transmission mechanism, the upper limit is set to the frame rate of the upper body image, and if 0 is described in the distance information 1001, 0 is set to the frame rate of the upper body image. Is done.
From the frame rate of the face image and the frame rate of the upper body image, the calling interval of the image transfer thread is obtained in milliseconds. The timer function of the information processing device 210 is set to call the image transfer thread for each value (step 1112).
[0038]
Through the above steps, setting information for transferring data from the transmission system is determined.
After the above data reception processing, until the next data 1000 comes from the display system 102, the imaging system 101 repeats the processing of the image transfer thread using these set values.
[0039]
Next, the operation of the image transfer thread will be described.
[0040]
The image transfer thread is activated by the timer function of the information processing device 210 set in step 1112. Hereinafter, a series of operations will be described for a case where a thread for transferring a face image is activated.
[0041]
In step 1201, input information input by the sender using the input device 209 is taken into the information processing device 210. This information is used as a script command for moving the avatar character or moving the foot. In step 1202, the information fetched in step 1201 is written to the script description area 902 of the data structure 900 for transfer. The character movement information is written in 903.
In steps 1203 to 1204, an image (face image) of the transmission user is created.
[0042]
As shown in FIG. 14, a table 1400 defining an area for capturing an image is defined in the information processing device 210 as a part of the program. The information in this table is composed of camera ID number C_ID (column 1401), rotation angle R (column 1406), and four numerical values w0, w1, h0, h1 (columns 1402 to 1405).
The ID of the camera selected in step 1109 is searched from column 1401. The information of the area linked to this value is read from the columns 1402 to 1405 of the table 1400 (step 1203).
In step 1204, an image of the camera corresponding to the camera number C_ID is read. This image is transferred to the information processing device 210 as electronic data.
[0043]
The image in the range of (w0, h0)-(w1, h1) among the images captured in step 1204 is an area to be transmitted as a transfer image. This range corresponds to the photographing region range 711 or 712 in FIG. Reference numeral 1501 in FIG. 15 represents an image in a range expressed by (w0, h0)-(w1, h1). This image 1501 is composed of pixels 1510. The pixel 1510 is constituted by three sets of numbers indicating a red value 1511, a green value 1512, and a blue value 1513, and a number 1514 indicating transparency. The resolution of the video in this area is reduced to the resolution calculated from the distance information 1001, and the video is written in the transfer data structure. For this purpose, for the h0th horizontal line, images of w0 to w1 are decimated using the information of D1 and D2 set in step 1110 as shown in FIG. (Step 1205). However, in the image 1501 of FIG. 15, the area is filled with black and is not transferred. The area to be transferred is the other area, and the image after the thinning is the image shown by the image 1502. In this transfer, if the color information indicated by the combination of the red value 1511, the green value 1512, and the blue value 1513 matches the color information of the chroma key color, the transparency information 1514 indicates transparency. Write the value. Otherwise, write a value indicating opacity.
[0044]
In step 1206, the image ID is written to the data area 901 of the data structure 900. This ID is one of fixed values (ID_FACE, ID_UPPER) representing any of the body part, the face, and the upper body. Further, the value of the horizontal width of the image 1502 is written in the data area 904, and the value of the vertical width of the image 1502 is written in the data area 905.
[0045]
The information of the transfer data structure 900 created in the above steps 1201 to 1206 is transmitted (step 1207).
[0046]
The display system receives the transfer data structure 900 (operation 1208).
[0047]
In step 1209, the position data of the avatar in the virtual space is moved based on the character movement data 903. In addition, the animation processing accompanying the movement is loaded.
[0048]
In step 1210, the three-dimensional shape model 703 constituting the lower body of the avatar performs an animation process associated with the script 902 in the action information area.
[0049]
If some animation processing has already been loaded, or if a walking animation for movement has been loaded, blending processing of these animations is performed. However, an existing method is used for the setting of the animation and the blending process ((Step 1211)).
[0050]
In step 1212, image data is read. The data 901 in the header area is checked. If it is ID_UPPER, it is determined that an upper body image has been sent. If it is ID_FACE, it is determined that a face image has been sent. Or on the texture area of the polygon 102.
[0051]
For the polygons whose texture has not been updated by this processing, rendering is performed using the same texture as the previous time.
[0052]
In step 1213, input information from the user 400 is read.
[0053]
The display system 401 acquires information input by the receiving user 400 to the input device 403 through the interface 503, and changes the viewpoint position 621 and angle 622 of the receiver in the virtual space by reflecting the input information.
[0054]
In step 1214, a screen is displayed. A rendering process is performed on the background image and the avatar model to create an image, and the image is displayed on the image display device 401. Also, at the time of this rendering processing, by performing rendering using the transparency information described in the image data, the background information of the person is correctly displayed.
[0055]
Thus, a series of operations of the image transfer thread ends.
[0056]
If the called image transfer thread is a thread for the upper body image, the table 1300 and the table 1400 are used for the upper body image, and the image is transferred to the texture area of the polygon 102 in step 1212. You. In other respects, the same steps as those of the face image transferring means described above are performed.
[0057]
In the above-described embodiment, an embodiment will be described in which a footage 103 of an avatar displayed using a three-dimensional shape model is replaced with a moving image of a previously shot video.
The configuration of the avatar character at this time is as shown in 1600 in FIG. The face part 1601, the upper body part 1602, and the lower body part 1603 are respectively created by billboards.
[0058]
Moving images obtained by capturing images of the lower body in each action such as stationary, walking, running, and sitting are stored in the storage device 504 in advance. In this video, a portion where a foot is shown is subjected to a stabilizing process so that each image looks the same size, and a background clipping process is performed. Further, image processing called morphing is performed on the video at the start and end portions of each moving image. By this moving image processing, when each moving image is reproduced continuously, it is displayed as a connected image.
[0059]
Also, a table 1700 shown in FIG. 17 is stored, and information for associating the moving image file 1620 with the action content and the shooting angle is defined. The character's foot 1603 is composed of billboard data.
In the above-described embodiment, the following processing is performed instead of the animation processing of the three-dimensional model performed in steps 1209 to 1211.
[0060]
First, a value described in the data 902 registered in the structure 900 is read. Save this value as the action value.
[0061]
Further, the value of the angle determined in step 1104 is referred to.
[0062]
The table 1700 is searched to read out from the column 1703 the file name of the data whose action ID data written in the column 1701 is the same as the current action value 902 and whose angle information in the column 1702 is closest to the reference angle. .
[0063]
On the billboard 1603, a video selected from the moving images 1610 to 1613 stored as local data is displayed.
[0064]
The moving image is repeatedly displayed until each information changes and another moving image is selected.
[0065]
In each of the above-described embodiments, a system that transmits images in both directions can be configured by combining the photographing system and the display system with each other.
[0066]
【The invention's effect】
According to the present invention, an avatar character whose angle and resolution are changed according to the position in the virtual space can be viewed.
[0067]
In addition, by using a polygon in the first embodiment and a moving image stored as local information in the second embodiment as a part of the display of the avatar character, a network generated with the transfer of the image is used. The amount of load can be reduced.
[0068]
Further, by controlling the frame rate and the resolution, the amount of load applied to the network when the system operates can be controlled.
[Brief description of the drawings]
FIG. 1 is a configuration of an entire system according to a first embodiment of the present invention.
FIG. 2 is a transmission system configuration.
FIG. 3 is a configuration of an information processing apparatus for a transmission system.
FIG. 4 is a configuration of a receiving system.
FIG. 5 is a configuration of an information processing apparatus for a transmission system.
FIG. 6 is a schematic diagram of a virtual space configuration.
FIG. 7 is a schematic diagram of an avatar configuration.
FIG. 8 is a configuration example of animation data.
FIG. 9 is a schematic diagram of a data structure for communication in an image transfer thread.
FIG. 10 is a schematic diagram of a communication data structure in an information acquisition thread.
FIG. 11 shows the operation of an information acquisition thread.
FIG. 12 shows the operation of an image transfer thread.
FIG. 13 is a table showing a relationship between avatar distance information, resolution thinning information, and data size.
FIG. 14 shows data indicating information on each camera and its trimming area and angle.
FIG. 15 is a schematic diagram for creating a transfer image from a captured image;
FIG. 16 is a schematic diagram of an avatar configuration according to a second embodiment.
FIG. 17 is a schematic diagram of a table for searching a moving image file name from each action and a display angle.
[Explanation of symbols]
101 photographing system, 102 display system, 103 network, 200 image transmission side user, 201 camera, 202 camera, 203 camera, 204 camera, 205 camera, 207 background chroma key cloth, 208 lower body part concealment chroma key cloth, 209 image transmission User input device, 210 Transmission system information processing device, 301 Central processing unit (CPU), 302 Main storage device, 303 Input / output interface device, 304 External storage device, 305 Image transmission user output device, 307 Network interface Device, 400 image receiving user, 401 image display device, 402 information processing device for receiving system, 403 image receiving user input device, 501 central processing unit (CPU), 502 main storage device
503 input / output interface device, 504 external storage device, 505 image receiving user output device, 507 network interface device, 601 virtual space, 602 background object, 603 background object, 610 transmission user avatar, 611 transmission user avatar position, 612 Sending user avatar direction, 620 Receiving user avatar, 621 Receiving user avatar position, 622 Receiving user avatar direction, 701 Face display billboard, 702 Upper body display billboard, 703 Lower body display polygon data
711 face image shooting area, 712 upper body shooting area, 800 animation data, 801 operation pattern ID, 802 number of basic frames, 803 time interval data between basic frames, 804 joint rotation information data, 900 data structure, 901 image type 902, an area for describing script information that describes an avatar animation operation, 903 an area for describing movement information for an avatar, 904 horizontal length information of a transferred image, 905 vertical length information of a transferred image, 906 image data to be transferred Area of
1000 data structure, 1001 avatar distance information, 1002 avatar angle information, 1101 calculation of relative position between avatars, 1102 calculation of relative angle, 1103 determination of whether avatar is within view of display position, 1104 sender Calculate the orientation and distance of the avatar pointing to 1105 Write to data structure 1000 1106 Process when avatar is not in view 1107 Transmit data structure 1108 Receive data structure 1109
Select the camera that shows the required angle
1110 Calculate resolution from distance information, 1111 calculate frame rate, 1112 set image transfer thread to timer, 1201 acquire input information from input device, 1202 write avatar operation instruction to data structure 900, 1203 camera for shooting Selection, 1204 shooting of an image by a camera, 1205 processing of extracting necessary parts from a shot image by trimming processing and thinning processing, 1206 writing of image data to data structure 900, transmission of 1207 data structure 900, 1208 data structure Receiving body 900, 1209 updating avatar character position and reading animation for walking, 1210 reading animation information, 1211 blending animation, 1212 reading images from data structure 900, 1212 3 Reading and processing of input information from input device 403, 1214 Rendering process and display of video, 1301 Numerical value of distance information is described in this column 1302 Numerical value of value D1 is described in this column 1303 This column describes the value of the value D1. 1302 This column describes the data size value after thinning out. 1401 This column describes the camera ID. 1402 This column describes The left end of the trimming is described. 1403 The right end of the trimming is described in this column. 1404 The upper end of the trimming is described in this column. 1405 The lower end of the trimming is described in this column. 1406 In this column, the angle of the user photographed by the camera is described. 1501 Image before thinning processing (trimmed photographed image)
1502 Image after thinning process (transfer image), 1510 Configuration of pixels constituting image, 1511 pixel red information, 1511 pixel green information, 1511 pixel blue information, 1511 pixel transparency information
1601 face display billboard, 1602 upper body display billboard, 1603 lower body display billboard, 1611 face image shooting area, 1612 upper body shooting area, 1620 lower body moving image corresponding to each action, 1701 action ID, 1702 angle information , 1703 File name of moving image file

Claims (3)

An image that captures the body image of the user is synthesized in a virtual space configured on a computer, and transmitted in a virtual space displayed on the receiver side in an image display device that performs communication at a remote place. An image to be transmitted based on the three-dimensional positional relationship of the sender, the required angle, resolution, frame rate, and information on the body part of the sender, sequentially transmitted to the sender, and transmitted based on the information transferred by the sender. A video presentation device having a function of switching the content of a video. An image that captures the user's physical information is synthesized in a virtual space configured on a computer, and transmitted in a virtual space displayed on the receiver side on an image display device that communicates at a remote place. The required angle, resolution, and frame rate information are sequentially transferred to the transmitting side based on the three-dimensional positional relationship of the user, and the image of the image transmitted based on the information is held locally by the receiving side. An image presentation device having a function of synthesizing with a video of a body part and displaying the image in a virtual space. An image that captures the user's physical information is synthesized in a virtual space configured on a computer, and transmitted in a virtual space displayed on the receiver side on an image display device that communicates at a remote place. The required angle, resolution, and frame rate information are sequentially transferred to the transmitting side based on the three-dimensional positional relationship of the user, and the image of the image transmitted based on the information is held locally by the receiving side. An image presentation device having a function of synthesizing with a three-dimensional shape model of a body part and displaying the same in a virtual space.

Images