JP2000098871A - Virtual image stereoscopic compositing device, virtual image stereoscopic compositing method, game device and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to easily recognize the virtual movements of an object existing on a reference plane of a real space with a small calculation quantity from the position informations of the object and to control the expression of the image of the virtual object composited on the reference plane of the virtual space in accordance with the position information thereof. SOLUTION: This device has a bracelet 1 for setting the reference plane which is used for recognizing the arbitrary reference plane on the real space, to which the observer belongs, a special glass 2 for compositing the image of a three-dimensional polygon 10 on the reference plane of the virtual space as if the three-dimensional polygon 10 exists in the position, to which the reference plane in the real space recognized by the bracelet 1 belongs, and at least a position detecting mechanism 4 for detecting the position information of an implement 30A for playing with a kitten composited in the position including the three-dimensional polygon 10 on the virtual space. Image processing is executed in such a manner that the character image in the virtual space in changed in accordance with the position information of the implement 30A for playing with a kitten obtained from the position detecting mechanism 4.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a virtual image three-dimensional synthesizing apparatus and a virtual image three-dimensional synthesizing method suitable for manufacturing a virtual character breeding game apparatus for growing TV program characters with palms in a virtual space. The present invention relates to a game device and a storage medium.

More specifically, a position detecting means is provided in the real space to which the observer belongs, and the position information of the object existing on the reference plane of the real space can be simply and with a small amount of calculation. It is possible to recognize virtual approach, contact or pressure, and to control the expression of the image of the virtual object synthesized on the reference plane of the virtual space based on the position information.

[0003]

2. Description of the Related Art In recent years, with the improvement of display technology based on virtual reality, a virtual image stereoscopic display device for providing a viewer with virtual reality over a plurality of image display surfaces has appeared. are doing.

[0004] This type of stereoscopic display device is disclosed in Japanese Unexamined Patent Publication No. 9-23.
7353 can be found in the technical literature. According to this technical document, a projection space of about several meters in length and width is provided, and a display device is arranged on each surface, and a dinosaur,
Images of virtual bodies such as monsters and weapons are displayed in three dimensions. Then, when the observer wears glasses with a liquid crystal shutter and stands in the projection space, it is as if he were at the same place as the virtual body displayed on each display device.

[0005] Further, a weapon that an observer holds in a virtual space is imaged by a camera, and image processing is performed so that a virtual body reacts according to the movement of the weapon. This allows the observer to slip back in time to the primitive era thousands of years ago, and perform dinosaur extermination, etc., as if playing a game.

[0006]

However, according to the conventional three-dimensional display device, the center of gravity of the three-dimensional shape is obtained on the assumption that the projection space has a three-dimensional shape, and the center of gravity of the three-dimensional shape is determined from the center of gravity. The coordinates are determined. Then, based on the relative coordinates, the positional relationship between the camera and the virtual body is obtained, and a virtual body such as a dinosaur is displayed in a composite manner in the virtual space.

[0007] For example, when an object is brought close to a virtual body such as a dinosaur virtually standing on a certain reference plane in the projection space, the approach, contact or pressing of the object to the virtual body on the reference plane is considered. When trying to identify, first, a rectangular area circumscribing the virtual body is extracted by image processing.

Thereafter, the positions of the four corners of the rectangular area on the image are obtained from the relative coordinates, and the camera and the positional relationship between the rectangular area of the virtual body are obtained using the parameters obtained by the perspective projection transformation method or the like. . Then, the position coordinates of the object with respect to the rectangular area are calculated, and the approach, contact or press of the object to the virtual body on the reference plane is identified.

[0009] Therefore, for example, a virtual image stereoscopic composition in which, for example, a TV program character or the like virtually jumps out onto the reference plane of the real space to which the observer belongs and the character plays with the palm of the observer in the virtual space. If a conventional stereoscopic display device is applied as it is when configuring the device, the device becomes large-scale, or image processing for calculating the approach, contact, or pressing of an object to a virtual object on a reference plane. Is complicated or the amount of calculation at that time is increased, leading to an increase in the cost of a game device or the like.

Accordingly, the present invention has been made in view of the above-described problems, and is intended to simplify the virtual motion of an object from the position information of the object existing on the reference plane in the real space with a small amount of calculation. Virtual image three-dimensional composition device, virtual image three-dimensional composition method, and game device capable of recognizing an image and controlling the expression of a virtual body image synthesized on a reference plane in a virtual space based on the position information thereof And a recording medium.

[0011]

An object of the present invention is to provide an apparatus for stereoscopically combining an image of a virtual object with an external image to which an observer belongs, and recognizing an arbitrary reference plane in a real space to which the observer belongs. Combining means for combining an image of a virtual body on a reference plane of a virtual space such that a virtual body exists at a position to which the reference plane in the real space belongs, which is recognized by the surface recognition means. And at least a position detecting means for detecting a position of an object synthesized in a position including the virtual body in the virtual space and located at a position distant from the reference plane in the real space, According to another aspect of the present invention, there is provided a virtual image three-dimensional composition device that performs image processing so that an image of a virtual body changes in a virtual space based on position information of an object obtained from a detection unit.

According to the virtual image three-dimensional synthesizing apparatus of the present invention,
When stereoscopically combining an image of a virtual body with an external image to which an observer belongs, a reference plane in a real space is obtained by a plane recognition unit. Then, the reference plane recognized by the plane recognition unit is displayed by the synthesis unit, and an image of the virtual body is synthesized in the virtual space to which the reference plane belongs. For an object such as an observer's hand, for example, synthesized at a position where the virtual body is arbitrarily touched on the display screen, the position of the object is detected in real space by the position detecting means. Image processing is performed so that the image of the virtual body changes in the virtual space based on the position information of the object.

Therefore, it is possible to easily obtain the displacement information such as the virtual approach, contact or press of the object in the virtual space from the position information of the object in the real space by the position detecting means with a small amount of calculation. Therefore, the virtual image three-dimensional composition device of the present invention can be sufficiently applied to a virtual character breeding game device and the like.

The virtual image stereoscopic synthesis method of the present invention is a method for stereoscopically synthesizing an image of a virtual body with an external image to which an observer belongs, and sets an arbitrary reference plane in a real space to which the observer belongs. By capturing the reference plane set here and synthesizing the image of the virtual body on the reference plane of the virtual space so that the virtual body exists at the position to which the reference plane in the real space belongs, at least the virtual In space, an object is synthesized at the position that includes the virtual body.In real space, the position information of the object located at a position distant from the reference plane is detected, and based on the position information of the detected object, the virtual The image processing is performed so that the image of the virtual object changes in space.

According to the virtual image three-dimensional synthesis method of the present invention,
Displacement information such as virtual approach, contact or pressure of the object in the virtual space can be easily acquired from the position change of the object in the real space with a small amount of calculation.

A game device according to the present invention is a device for stereoscopically combining an arbitrary character image with an external image to which an observer belongs, and a plane recognition means for recognizing an arbitrary reference plane in a real space to which the observer belongs. Synthesizing means for synthesizing an image of the character on the reference plane of the virtual space so that the character exists at the position to which the reference plane in the real space belongs, which is recognized by the plane recognition means; Above, an object attached to a part of the body part of the observer or an object held by the observer to be combined with the position including the character, and the position of the object in the real space that is away from the reference plane Position detecting means for detecting information, wherein image processing is performed so that a character image in a virtual space changes based on position information of the object obtained from the position detecting means. Than it is.

According to the game apparatus of the present invention, since the above-described virtual image three-dimensional composition apparatus is applied, for example, a data glove or an observation device attached to a part of the body part of the observer in the real space by the position detection means Displacement information such as virtually approaching, touching, or pressing a data glove or a clasp in a virtual space can be easily acquired from positional information of a clasp and the like held by the user with a small amount of calculation.

Therefore, a data glove in which a character or the like of a TV program jumps out to the position to which the reference plane in the real space belongs, and a character or the like placed on one palm of the observer is mounted on the other hand, The character can be made to laugh, cry, or be angry when the pet is brought close to the stirrer and stroked. This makes it possible to inexpensively configure a virtual character breeding game device or the like in which the observer breeds a character that appears as if it appeared at the position to which the reference plane belongs in the real space.

The recording medium of the present invention is a recording medium storing an algorithm for stereoscopically combining an image of a virtual object with an external image to which an observer belongs, and the recording medium has a structure in real space to which the observer belongs. An arbitrary reference plane is set, and the reference plane set here is imaged and displayed in the virtual space, and the image of the virtual body is superimposed and synthesized on the virtual space. Detects the position information of an object that is synthesized at the position that encompasses the body and is located away from the reference plane in the real space, and based on the detected position information of the object, the virtual information in the virtual space is detected. An algorithm for performing image processing such that a body image changes is stored.

According to the recording medium of the present invention, the algorithm for executing the above-described virtual image three-dimensional synthesis method is stored. By executing the algorithm, the position information of the object in the real space can be easily obtained. Thus, it is possible to recognize a virtual approach, contact or pressure of an object in a virtual space with a small amount of calculation. Therefore, the above-described virtual character breeding game device and the like can be configured with good reproducibility.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a virtual image three-dimensional composition device, a virtual image three-dimensional composition method, a game device, and a recording medium according to embodiments of the present invention will be described with reference to the drawings.

(1) First Embodiment FIG. 1 is a perspective view showing a configuration example of a game apparatus 100 to which a virtual image stereoscopic synthesis apparatus according to a first embodiment of the present invention is applied.

In this embodiment, a position detection is performed to detect a position of an object which is synthesized at a position touching a virtual body such as a character in a virtual space, and which is located at a position distant from a reference plane in a real space. Means for performing image processing so that the image of the virtual body in the virtual space changes based on the position information of the object detected here, and as if the TV program is located at the position to which the reference plane in the real space belongs. A virtual character nurturing game device that makes a character laugh, cry, or angry when a character or the like jumps out and strokes a character placed on one palm of the observer with the other hand approached. And so on can be configured inexpensively and compactly.

The game apparatus 100 shown in FIG. 1 is an apparatus for stereoscopically combining a virtual body image such as a TV program character with an external image to which an observer belongs. This game device 100
Has a reference surface setting bracelet 1, a special glasstron 2, an image processing device 3, and a position detection mechanism 4.
The bracelet 1 is used, for example, mounted on the left hand of the observer. In this example, when the object 30 is brought close to the bracelet 1, the position detection mechanism 4 detects the distance Sx between the reference surface of the bracelet 1 and the object 30.

In this example, the object 30 is assumed to be a jig held by the right hand of the observer, but is not limited to this, and may be a data glove. Two light-emitting diodes LED11 and LED12 are mounted at arbitrary positions as two or more light sources on this baffle, and are blinked in different blinking patterns.

The special glasstron 2 constitutes a synthesizing means. A belt 22 is provided on the main body 21 of the special glasstron 2. The main body 21 is attached to the face of the observer as if wearing glasses, The belt 22 is fixed along the outer periphery of the head. The special glasstron 2 is provided with at least a panning CCD device 23 and a display unit 24. Normal CCD depending on the model of special glasstron 2
An imaging device 25 is provided. The position detecting mechanism 4 is constituted by the object 30 such as the above-mentioned buzzer and the panning CCD device 23, and the object 30
Is detected.

Further, the bracelet 1, the panning CCD device 23 and the image processing device 3 constitute a surface recognizing means so that an arbitrary reference plane can be recognized in the real space to which the observer belongs. Here, the follow shot means a shooting mode in which signal charges are read out from a photoelectric conversion element (such as a photodiode) a plurality of times in the same field period in the follow shot CCD device 23.

In this example, when a two-dimensional imaging device of an interline transfer system having a vertical transfer unit is used as the panning CCD device 23, a signal is sent from the photoelectric conversion element to the vertical transfer unit a plurality of times during the same field period. The charge is read. When a two-dimensional frame transfer type imaging device having a charge storage unit is used as the panning CCD device 23, signal charges are read from the photoelectric conversion element to the charge storage unit a plurality of times during the same field period.
Therefore, in this example, the panning CCD device 23 is shared by the position detection mechanism 4 and the surface recognition means.

Further, an image processing device 3 is connected to the special glasstron 2 so that the image of the virtual body changes in the virtual space based on at least the position information of the object 30 obtained from the position detection mechanism 4. Image processing is performed. The image processing device 3 performs image processing for recognizing a reference plane or the like based on image data output from the panning CCD device 23. The display unit 24 is connected to the image processing apparatus 3 and displays the reference plane recognized by the plane recognition unit.

An optical means such as a polarizing beam splitter may be provided in the special glasstron 2, and an image of a virtual body is synthesized on a reference plane of a virtual space displayed by the display means 24. In this example, a 3D polygon (snowman) 10 as a virtual body is present at the position to which the reference plane in the real space belongs.

In this example, a reference surface setting bracelet 1
Has a plate portion 11 and a bracelet portion 12 shown in FIG. The plate portion 11 forms a reference surface, and is formed in a flat shape without irregularities. The size of the plate portion 11 is, for example, about the size of a wristwatch, and the length of one side thereof is about 3 cm, and at most about 5 cm. Light emitting diodes (LED1 to LED4) are attached to four corners of the surface of the plate portion 11 as three or more light sources, respectively.
(X1, y1), (x2, y) as coordinates of four points P1 to P4 of the reference plane from which the D polygon 10 is to be protruded.
2), (x3, y3) and (x4, y4) are given (corresponding to a reference plane on which an image is to be synthesized in the virtual space).

The four light emitting diodes LED1 to LED4
Is blinked so as to exhibit its function as a mark portion, that is, at least in a different blinking pattern so that its mounting position is clear. The light emitted from the light emitting diodes LED1 to LED4 is imaged by the panning CCD device 23 in the special glasstron 2 so as to flow in a predetermined imaging direction. This panning is for specifying the reference plane from the mounting positions of the four light emitting diodes LED1 to LED4. This reference plane is specified in FIGS. 9 and 11.
The explanation is given below.

The bracelet 12 preferably has a structure that can be extended and contracted so as to fit the wrist of the observer. For example,
For example, a Velcro is provided on the band member so that the mounted state can be adjusted. The plate portion 11 is preferably mounted on the back side of the left hand so as to be within the shooting range of the panning CCD device 23.

The special glasstron 2 shown in FIG. 3 constitutes a non-transmissive head mounted display, and includes a normal CCD image pickup device 25 and the above-mentioned panning CCD device 2.
3, a right-eye display liquid crystal display (hereinafter referred to as LCD) 26 as a first image display element, and a left-eye display LCD 27 as a second image display element.

That is, a normal CCD image pickup device 25 and a panning CCD device 23 are arranged side by side on the main body 21 at a position corresponding to the eyebrows of the observer, and the former captures an external image to which the observer belongs. Then, the four light emitting diodes LED1 to LED4 of the bracelet 1 are shot by the latter. Therefore, when the observer looks at the above-mentioned bracelet 1, the panning CCD device 23 and the CCD imaging device 25 are oriented in the direction of the reference plane. In addition, L is located at a position facing the right eye of the observer in the special glasstron 2.
A CD 26 is attached, for example, an observer's bracelet 1 photographed by a normal CCD image pickup device 25, an object 30 held by the observer, and an image of a 3D polygon 10 prepared by computer graphics (CG) prepared in advance. Is displayed.

An LCD 27 is attached at a position facing the left eye of the observer, and the other of the above-mentioned stereo images is displayed. The special glasstron 2 is mounted on the face or head of the observer, and superimposes the stereo image on the LCD 26 and the stereo image on the LCD 27 so as to guide the stereo image to the eyeball of the observer. Thus, the background image to which the observer belongs and the 3D polygon 10 are combined in the head.

The special glasstron 20 shown in FIG. 4 constitutes a transmission type head mounted display, and does not have a normal CCD image pickup device 25 mounted thereon. Accordingly, the transmissive head-mounted display includes the panning CCD device 23, a liquid crystal shutter 28 for capturing an external image, and an LCD 29 as an image display element.

For example, a panning CCD device 23 is disposed in the main body 21 at a position corresponding to the eyebrow of the observer. When the observer turns his / her eyes to the bracelet 1, the position of the object on the reference plane is determined. Detected and 4 of bracelet 1
The light-emitting diodes LED1 to LED4 are panned. Liquid crystal shutters 28 are provided at positions corresponding to the left and right eyes of the observer. For example, when the liquid crystal shutter 28 is opened, the real image of the bracelet 1 of the observer passing through the liquid crystal shutter 28 is directly guided to the eyeball. I will

An LCD 29 is attached to a portion of the special glasstron 2 located beside the left or right eye of the observer, and a character image is displayed in the same manner as the special glasstron 2 described above. Although not shown, the liquid crystal shutter 2
Optical means such as a polarization beam splitter is provided between the LCD 8 and the LCD 29 so that the real image of the bracelet 1 of the observer and the image of the 3D polygon 10 are guided to the eyeball of the observer. Thus, the background image to which the observer belongs and the 3D polygon 10 are combined in the head.

Next, the internal structure of the follow-up CCD device 23 of the interline transfer system will be described. The panning CCD device 23 shown in FIG.
On the substrate 31, photodiodes PHij (i = 1 to n, j = 1 to 1) as photoelectric conversion elements forming one pixel are provided.
m) are arranged in a matrix of n columns × m rows.

In the column direction of this substrate, m
Vertical transfer units 32 are provided, and the photodiodes PH
The signal charge read from ij is transferred in the vertical direction (follow-up direction) based on the vertical read signal S1. A horizontal transfer unit 33 is connected to the vertical transfer unit 32, and the signal charges are transferred in the horizontal direction based on the horizontal readout signal S2, so that a follow shot signal SOUT is output to the output terminal. In this example, the signal charge is transferred from the photodiode PHij to the vertical transfer unit 32 at least a plurality of times during the same field period in order to perform a panning shot.

The panning CCD device 23 has a fisheye lens 35 shown in FIG. The fisheye lens 35 is provided on the optical axis of the CCD image sensor 36, for example. The fisheye lens 35 allows the observer to image the above-mentioned bracelet 1 and the like in a wide range. Of course, a normal lens may be used, but since the field of view is narrowed, the observer must tilt his head more toward the bracelet 1.

Subsequently, the shaver 30 as the object 30
Configuration examples of A, 30B, data glove 30C, and force sense glove 30D will be described. FIG. 7A shows a jig 30
It is a front view which shows A. The clasp 30A includes a spike-shaped portion 301 and a hand portion 30 formed by injection molding a resin, for example.
Two. Arbitrary position Q of spike 301
A light emitting diode LED11 is attached to 1 and a light emitting diode LED12 is attached to the position Q2.

A flashing control circuit (not shown) is provided in the spike portion 301 of the bat, and a predetermined voltage is applied to the two light emitting diodes LED11 and LED12 to perform a flashing control for panning. In this blinking control, for example, the light emitting diode LED1
1 and the blinking interval of the LED 12 are controlled. A small dry cell or button cell (not shown) is used as a power source for the light emitting diodes LED11 and LED12.

FIG. 7B is a front view showing a bird feather-like baffle 30B. The baffle 30B is also made of a resin molded member and has two light emitting diodes LED13 and L
The ED 14 is attached, and a predetermined voltage is applied and blinking control is performed for panning in the same manner as the hanger 30A.

FIG. 8 shows a jig 3 using an optical fiber.
It is a conceptual diagram which shows the example of an internal structure of 0A. The harness 30A shown in FIG. A printed board 312 is provided on the right side of the housing 311.
2 is provided with a driver IC 313 constituting a blinking control circuit with three light emitting diodes LED0 to LED2.

Near the center of the housing 311, the light emitting window 3
15 are formed, and about several tens of optical fibers 316 are exposed. The light emitting diode LED0 is connected to the other end of the optical fiber 316, and is constantly lit by the driver IC 313 so that an illumination effect can be obtained.

The housing 311 is provided with a light emitting window 314 at the position Q1, and the ends of several optical fibers 317 are exposed. A light emitting diode LED11 is connected to the other end of the optical fiber 317, and a driver IC 313 is provided.
Controls the blinking pattern. Furthermore, this case 3
A light emitting window 316 at a position Q2 is provided at 11, and the ends of several optical fibers 318 are exposed. The other end of the optical fiber 318 is connected to the light emitting diode LED12, and the driver IC 313 controls the light emitting diode L12.
Control is performed so that the blinking pattern differs from that of the ED 11.

When such optical fibers 316 to 318 are used, it is possible to form the jig 30A compactly and to make the jig 30A that is optically brilliant and beautiful. Shroud 3
0B can be similarly configured.

FIG. 9 is a front view showing the data glove 30C. The data glove 30C is for outputting pressure distribution information when the object is touched, and has, for example, a glove part 303 shaped like a right hand. In this example, seven light emitting diodes LED21 to LED27 are connected to the glove part 303.
Attached along the instep side. This data glove 3
A blinking control circuit (not shown) is connected to 0C, and each of the light emitting diodes LED21 to LED21 is used for panning.
A predetermined voltage is applied to 27 to perform blink control. In this blink control, the blink interval is controlled so that the blink pattern is different. A small dry cell or button cell (not shown) is used as a power source for the light emitting diodes LED21 to LED27.

FIG. 10 is a front view showing a force sense presentation glove 30D. The force sense presentation glove 30D is configured to output a touch sense when touching an object as a force sense presentation signal S0.
04. In this example, about 20 light emitting diodes LED are attached along the five fingers and the back side of the glove part 304. A blink control circuit (not shown) is connected to the globe 30D, and a predetermined voltage is applied to each of the light emitting diodes LED to perform blink control for panning.

When such a data glove 30C or a haptic presentation glove 30D is used, pressure distribution information and a haptic presentation signal S0 are directly obtained, and the information and the signal S0 are obtained.
Image processing for changing the expression of the 3D polygon 10 based on the

Next, the circuit configuration of the game device 100 will be described. The game device 100 shown in FIG. 11 is roughly composed of three circuit blocks. The first circuit block is the above-described bracelet 1, and this bracelet 1
Is provided with a blink control circuit 5, and a predetermined voltage is applied to the four light emitting diodes LED1 to LED4 to perform blink control. In the blinking control circuit 5, the blinking patterns are different.
For example, light emitting diodes LED1, LED2, LED3
And the blinking interval of the LED 4 is controlled. A small dry battery or button battery (not shown) is used as a power source for the light emitting diodes LED1 to LED4.

The second circuit block is a special glasstron 2
In the case of the non-transmissive type, the panning CCD device 23 described above, the normal CCD image capturing device 25, and the LC for displaying the right eye are used.
D26 and LCD 27 for left eye display.

The third circuit block is the image processing device 3 and has an internal bus 41. The internal bus 41 includes an interface (I / O) 42, an image capture unit 43, an image processing unit 44, a CPU 45, a ROM 46, a RAM 47, and an E ² PROM (read only memory capable of writing and erasing electrical information) 48. It is connected. The panning CCD device 23, the normal CCD imaging device 25, the LCD 26 for right-eye display, and the LCD 27 for left-eye display are connected to the internal bus 41 via the interface 42.

The internal bus 41 has E ² as a recording medium.
The PROM 48 is connected, and the 3D
An algorithm for three-dimensionally synthesizing the image of the polygon 10 is stored.

For example, the E ² PROM 48 captures an arbitrarily set reference plane in the real space to which the observer belongs, displays the reference plane in the virtual space, and displays the 3D polygon 10 in the virtual space. Are synthesized at the position including the 3D polygon 10 in the virtual space.
In the real space, the object 30 located at a position away from the reference plane
, An algorithm for detecting the position of the object 30 on the reference plane in the real space and performing image processing such that the image of the 3D polygon 10 in the virtual space changes based on the position information Sx of the detected object 30 is stored. Is done.

Therefore, by executing this algorithm, the object 30 on the reference plane of the real space to which the observer belongs
Can be easily recognized with a small amount of calculation. Accordingly, image processing can be performed as if the 3D polygon 10 exists at the position to which the reference plane in the real space belongs, so that a character or the like that appears on the reference plane in the real space moves according to the instruction of the observer. It is possible to configure the virtual character breeding game device and the like with good reproducibility.

Further, a ROM 46 is connected to the internal bus 41, and stores a system program for controlling the game apparatus 100, control information such as a memory reading procedure, and the like. The internal bus 41 has a working RA
M47 is connected, and display information for displaying a system program and a character image is temporarily recorded. A CPU 45 is connected to the internal bus 41,
2. Image capture unit 43, image processing unit 44, ROM4
6, control of the input and output of the RAM 47 and the E ² PROM 48, the panning CCD device 23, the CCD imaging device 25, the L
Input / output control of the CD 26 and the LCD 27 is performed.

The interface 42 has an image processing unit 4
4 is connected, and the image of the above-described bracelet 1 shown in FIG.
The image data is taken into the image processing unit 44 via the interface 42 together with the control command of No. 5, where predetermined image processing is performed, and is transferred to the LCD 26 in the special glasstron 2 via the interface 42 again.

An image capture unit 43 is connected to the interface 42. The control unit 45 receives a control command from the CPU 45 and blinks the four light emitting diodes LED1 to LED4 input from the panning CCD device 23 and the two light emitting diodes LED11. A predetermined capture process is performed to acquire image data relating to and the blink pattern of the LED 12. The image data of this blinking pattern is represented as a change in luminance corresponding to the passage of time. An image processing unit 44 is connected to the image capturing unit 43 via the internal bus 41, and regarding image data on which predetermined image processing has been performed,
The synchronization deviation of the blinking pattern described above is corrected, or a reference plane to which the observer belongs is obtained.

For example, in the image processing section 44, the panning CC
Panning signal (luminance signal) S output from D device 23
Regarding the blinking pattern of OUT, in the image area defined by the window W shown in FIG. 12, an XY plane including four follow-up luminous points P1 to P4, and two follow-up luminous points Q1 and Q2 on the XY plane. Is converted to a spatial arrangement pattern Then, by scanning the arrangement pattern, at least the position coordinates (X
1, Y1), (X2, Y2), (X3, Y3), (X
4, Y4) and the position coordinates (X) of the two bright points Q1 and Q2.
5, Y5) and (X6, Y6).

The four bright points P1 to P4 correspond to the four light emitting diodes L of the bracelet 1 attached to the observer.
ED1 to ED4, and the two bright spots Q1 and Q2 correspond to the two light emitting diodes LE of the harness 30A held by the observer.
D11 and LED12. The position coordinates (X5, Y5) and (X6, Y6) of the bright spots Q1 and Q2 are the position information S.
x. Four light emitting diodes LED1 through real space
4 is known, and its position coordinates are (x1, y
1), (x2, y2), (x3, y3), (x4, y
4).

Therefore, the above-mentioned reference plane in the real space can be obtained by calculating a conversion matrix projected onto the mounting positions of the four light emitting diodes LED1 to LED4. Here, the point (xi, yi, 0) on the plane of the real space is moved by a certain translation / rotational motion, and the point projected on the image coordinate system by the perspective transformation is represented by (Xi, Yi). The following equation (1) holds between them.

[0065]

(Equation 1)

Where a1... A8 are unknown coefficients and C
External parameters (position and direction) such as CD imaging device 25
And internal parameters such as focal length. These parameters are the position coordinates (x1, y) of a known point in the real space.
1), (x2, y2), (x3, y3), (x4, y
4) and four sets of position coordinates (X1, Y1), (Y2, Y2), (X3, Y3),
If (X4, Y4) exists, it can be obtained by solving the following equation (2).

[0067]

(Equation 2)

The position coordinates (X1, Y
1), (X2, Y2), (X3, Y3), (X4, Y
By connecting 4), the reference plane in the real space shown in FIG. 12 is recognized. The buzzer 30A on the reference plane has the position coordinates (X5, Y5), (X6, Y5) of the bright spots Q1 and Q2.
It is recognized as a line segment connecting 6).

More specifically, when the moving image pickup direction is the Y axis on the arrangement pattern shown in FIG. 13 and the direction orthogonal to the Y axis is the X axis, the image processing unit 44 sets the same direction as the moving image pickup direction. Alternatively, the luminance signal values are added in the opposite direction.
When this added value is plotted on the X axis, six positions where the luminance signal value plotted on the X axis is maximum are detected, and X coordinate values X1, X2, and X corresponding to the six positions are detected.
X3, X4, X5 and X6 are determined. Further, when the acquired image is scanned in the Y direction on the arrangement pattern, among the plurality of luminescent points arranged in the Y direction, the luminescent spot position that first emits light has a Y coordinate value Y1 corresponding to the X coordinate value. , Y2, Y
3, Y4, Y5, and Y6.

Here, the position coordinates of the four light emitting diodes LED1 to LED4 in the real space are defined as wi (i = 1 to 4), and the position coordinates wi of the four light emitting diodes LED1 to LED4 on the camera coordinate system. Assuming that the expression vector is Ci, the position coordinates of the four light-emitting diodes LED1 to LED4 on the LCD screen are Pi, the rotation matrix of the panning CCD device 23 is R, and the movement vector is T, the following equation (3) is obtained.
That is, Ci = R · wi + T (3) where Ci = Pi · ki (ki is a scalar). Therefore, the normal CCD imaging device 25
Of the rotation matrix R and its movement vector T,
Using this as a parameter, coordinate conversion between the real space and the virtual space can be easily performed, so that a character image can be synthesized on a reference plane in the virtual space.

Further, the image processing section 44 in this example performs image processing based on the position information Sx by the position detecting mechanism 4. For example, the above-described ROM 46, RAM 47 or E
^{2 In the} PROM 48, 3
Display information D1 to D3 for moving the image of the D polygon 10 is stored, and based on the position information Sx detected by the position detection mechanism 4, the ROM 46, the RAM 47, or the E
^{2 The} display information D1 to D3 may be read from the PROM 48.

FIG. 14 is a conceptual diagram showing an example of display information stored in the E ² PROM 48. In this example, at least the first to third reference position information is stored in the E ² PROM 48.

Here, the image processing unit 44 calculates a distance Sx from the reference plane to a line connecting the positions Q1 and Q2 of the buzzer 30A, for example. In this example, the position coordinates (X5, Y5) and (X6, Y6) of the bright points Q1 and Q2 are the position information Sx. Accordingly, assuming that the height h of the 3D polygon 10 is Sx> h as the position information Sx included in the reference position information, the baffle 30A is set to 3
Since a state in which the 3D polygon approaches the D polygon 10 can be assumed, display information D1 that makes the 3D polygon cry is read.

When Sx = h is detected as the position information Sx included in the second reference position information, it is possible to assume a state in which the buzzer 30A comes into contact with the 3D polygon 10. Is read out so as to laugh. When Sx <h is detected as the position information Sx included in the third reference position information, a state in which the buzzer 30A presses the 3D polygon 10 can be assumed.
The display information D3 that makes 0 angry is read. With the display information D1 to D3, the expression of the 3D polygon 10 on the reference plane can be changed.

Next, the operation of the game apparatus 100 with respect to the virtual image stereoscopic composition method of the present invention will be described. In this example,
It is assumed that the expression of the image of the 3D polygon 10 that is three-dimensionally synthesized with the external image to which the observer belongs is changed. At this time, the object 30 to be brought close to the 3D polygon 10 is a flap 30A held by the observer in the right hand.

For example, the observer wears the special glasstron 2 shown in FIG. 3 on the head. First, in step A1 of the flowchart shown in FIG. 13, in order to set an arbitrary reference plane in the real space to which the observer belongs, the observer, for example, sets the reference plane on the left arm so that the plate unit 11 faces upward. The bracelet 1 for use. Thereafter, the blink control circuit 5 is turned on to blink the four light emitting diodes LED1 to LED4 in a predetermined blink pattern.

Next, in step A 2, on the other hand, the reference plane in the real space is photographed using the ordinary CCD image pickup device 25 and displayed on the LCD 26. On the other hand, panning CCD
Using the device 23, a reference plane in the real space is shot.
For example, four light emitting diodes LED1 to LED4 attached at positions where images of the 3D polygon 10 are to be synthesized.
Are blinked so that the blinking pattern is different, so that the blinking pattern is imaged so as to flow in a predetermined imaging direction.

Thereafter, in step A3, image processing is performed to recognize an arbitrarily set reference plane in the real space to which the observer belongs. In the image processing unit 44, for example, a subroutine shown in FIG. 16 is called, and a video capture process is executed in step B1. Then, in step B2, the light emitting diodes LED1 to LED4 at the four corners are recognized. Specifically, the blinking pattern of the luminance signal by the four light emitting diodes LED1 to LED4 captured by the panning CCD device 23 is converted into a spatial arrangement pattern forming an XY plane including the four bright points P1 to P4. You.

After that, the arrangement pattern is scanned, and
At least the position coordinates (X1,
Y1), (X2, Y2), (X3, Y3), (X4, Y
4) is calculated, and the above-described expressions (1) and (2) are calculated, and the mounting positions of the four light emitting diodes LED1 to LED4 in the real space and the position coordinates (X1, Y) of the four points of the image processing system.
1), (X2, Y2), (X3, Y3), (X4, Y
4) is obtained, and a reference plane is obtained by connecting these four points. Then, in step B3, the image processing unit 44 performs an arithmetic process based on the above equation (3).
The positional relationship between the panning CCD device 23 and the reference plane is detected. Based on the above equation (3), the image of the 3D polygon 10 is synthesized on the reference plane in the virtual space.

At this time, in the special glasstron 2 worn by the observer, one of the stereo images obtained by synthesizing the external world image in the real space by the LCD 26 and the image of the 3D polygon 10 is guided to the right eyeball of the observer. You. The other of the stereo images by the LCD 27 is directed to the left eyeball of the observer. As a result, the background image in the real space to which the observer belongs and the 3D polygon 10 appearing in the virtual space are synthesized in the head, and it is as if the 3D polygon is located at the position to which the reference plane in the real space belongs. 10 can be present.

Thereafter, when the observer approaches the brace 30A held by the right hand on the reference surface, the process returns to the main routine of FIG. 15, and in step A4, the position detection mechanism 4 of the special glasstron 2 causes The position information Sx of the upper baffle 30A is detected. Of course, the two light emitting diodes LE are controlled by a blink control circuit (not shown).
D11 and LED 12 blink in a predetermined blink pattern. This blinking pattern is input to the image processing unit 44.
The image processing unit 44 calls a subroutine shown in FIG. 16 and executes a video capture process in step B1.

Then, in step B2, the LEs at the previous four corners are set.
Similarly to the recognition process of D, the two light emitting diodes LED11 and LED12 of the shroud 30A are recognized. Specifically, the blinking pattern of the luminance signal by the two light emitting diodes LED11 to LED12 captured by the panning CCD device 23 is XY including two bright points Q1 and Q2.
It is converted into a spatial arrangement pattern that forms a plane.

Thereafter, the arrangement pattern is scanned, and
Position coordinates (X5, Y5) of the two bright points Q1 and Q2,
(X6, Y6) are obtained, and the position coordinates (X
5, Y5) and (X6, Y6) are obtained as position information Sx of the buzzer 30A.

Thereafter, the process returns to the main routine of FIG. 15, and in step A5, an image of the 3D polygon 10 is synthesized on the reference plane of the virtual space based on the position information Sx.
For example, by moving the observer's right hand up and down,
By moving A, if the position information Sx of the baffle 30A is at a distance close to the 3D polygon 10 in the virtual space, Sx is used as the position information included in the reference position information shown in FIG. > H is detected, the display information D1 is read from the E ² PROM 48,
The LCD 27 can change the expression to make the 3D polygon cry based on the display information D1.

When the buzzer 30A shown in FIG. 17B is at a distance such that it comes into contact with the 3D polygon 10 in the virtual space, Sx = h is used as the position information included in the reference position information shown in FIG. Is detected, E ² P
The display information D2 is read from the ROM 48 and
6. The LCD 27 can change the 3D polygon 10 into an expression that makes it laugh based on the display information D2.

Further, when the jig 30A held by the right hand of the observer is at a distance that presses the 3D polygon 10 in the virtual space, the position information included in the reference position information shown in FIG. Since Sx <h is detected, the display information D3 is read from the E ² PROM 48,
The LCD 26 and the LCD 27 can change the 3D polygon 10 to an angry expression based on the display information D3. Therefore, although the 3D polygon 10 does not appear on the reference plane in the real space shown in FIG.
In the virtual space shown in FIG. 7B, a jig 30A holding the 3D polygon 10 appearing on the reference plane in the right hand of the observer.
Can be petted.

As described above, according to the game apparatus 100 to which the virtual image three-dimensional synthesizing apparatus according to the first embodiment is applied, when the character image of the TV program is stereoscopically synthesized with the external image to which the observer belongs. The position information S, such as a buzzer 30A held in the right hand of the observer, synthesized at a position including the 3D polygon 10 in the virtual space.
x can be easily acquired by the position detection mechanism 4 with a small amount of calculation.

Therefore, based on the position information Sx obtained from the position detecting mechanism 4, the 3D polygon 10
Can be changed so that a TV program character or the like jumps out to the position to which the reference plane in the real space belongs, and a character or the like placed on one palm of the observer on the other hand. When the user strokes the character close to the hand, the character can be made to laugh, cry, or make the character angry. This allows
A virtual character breeding game device for breeding characters and the like in a virtual space can be configured at low cost.

(2) Second Embodiment FIG. 18 is a diagram showing an example of a two-dimensional barcode 50 for setting a reference plane used in a game device according to a second embodiment. In this embodiment, four light emitting diodes LED1 to
The reference plane is recognized using a two-dimensional barcode 50 shown in FIG.

The surface recognizing means of this example has a two-dimensional matrix code 50 and is used for recognizing a reference plane in a real space. The two-dimensional barcode 50 is at least
A black and white matrix of n rows × n columns printed in black on a white background,
It is composed of a black frame part 51 having the same thickness as the black and white matrix.
In this example, 5 × 5 pixels surrounded by a black frame portion 51 are a code region portion 52, and among the 25 pixels, one shown in FIG.
Two pixels are painted black.

The bar code 50 is attached to a position where an image of the 3D polygon 10 is to be synthesized, for example, the surface of the plate portion 11 of the bracelet 1 shown in FIG. This two-dimensional matrix code 50 is panning CC
The image is picked up by a normal CCD image pickup device 25 instead of the D device 23. The image processing device 3 described with reference to FIG. 11 is connected to the output stage of the imaging device 25. The image processing device 3 is provided with arithmetic means, and performs image processing on a normal image pickup signal (luminance signal) output from the CCD image pickup device 25 to obtain a reference plane from the two-dimensional matrix code 50.

For example, the image processing section 44 performs preprocessing. In this process, first, the acquired image is set at 2 with an appropriate threshold.
Valued. Since the barcode portion is printed in black on a white background, the background image and the code area can be separated quite stably by the fixed threshold value. Next, labeling is performed for each connected region of black pixels. The black frame portion 51 of the two-dimensional barcode 50 is included in any of the labeled connection regions. Therefore, considering the size and aspect ratio of the circumscribed rectangle of the connection area, the code area 52
The background image (region) that is unlikely to include the symbol is removed.

Thereafter, a barcode frame is applied to each element of the connected area obtained as a result of the preprocessing. For example, a black region is searched from each side of the circumscribed rectangle toward the inside, and a point sequence of the black frame portion 51 is obtained. A line segment is applied to this point sequence by the least squares method. Then, the code area section 52 given to the two-dimensional barcode 50 is recognized.

The above-mentioned reference plane is obtained by calculating a transformation matrix for projecting the four vertices of the black frame 51 to the vertices of a square. Here, the point (xi, yi, 0) on the plane of the real space is moved by a certain translation / rotational motion, and the point projected on the image coordinate system by the perspective transformation is represented by (Xi, Yi). There is a similar relationship between the expressions (1) described in the first embodiment.

Accordingly, those parameters are the position coordinates (x1, y1), (x2, y2),
(X3, y3), (x4, y4) and their corresponding four sets of position coordinates (X1, Y1), (Y2,
If (Y2), (X3, Y3) and (X4, Y4) exist, they can be obtained by solving the equation of the above-described equation (2).

The position coordinates (x1, y1) obtained here,
Regarding (x2, y2), (x3, y3), and (x4, y4), assuming that four vertices of a square whose side length is “1”, a plane connecting these four vertices is a reference plane in the real space. Become. Although the black frame portion 51 on the screen is distorted by the attitude of the CCD imaging device 25 and perspective projection, a rectangular vertex on the screen can be projected to each square vertex by an external parameter and an internal parameter. Therefore, since the cubic 53 can be created from the position coordinates of the four corners of the two-dimensional barcode 50 in the virtual space shown in FIG. 19, the 3D polygon 10 and the like can be combined with the cubic 53.

As described above, according to the game device of the present embodiment, the 3D polygon 10
When the three-dimensional image is synthesized three-dimensionally, the two-dimensional bar code 50
Can be easily photographed, as in the first embodiment, by simply taking a reference plane in the real space to which the observer belongs.
It can be recognized with a small amount of calculation. Further, based on the position information Sx obtained by the position detection mechanism 4, image processing can be performed so that the image of the 3D polygon 10 changes in the virtual space.

Therefore, similarly to the first embodiment, the position of the reference plane in the real space to which the observer belongs is as if T
It is possible to configure a virtual character area-based game device or another game device that allows characters of a V program to jump out and play with the palm of the observer.

In each embodiment, the case where the non-transmission type special glasstron 2 or the transmission type special glasstron 20 is used has been described. However, the present invention is not limited to this, and the transmission type and the non-transmission type can be switched. Of course, it is possible to use a special type of glasstron which can be used for both purposes.

In each embodiment, the case where the 3D polygon 10 is simply projected virtually on the reference plane has been described. However, the present invention is not limited to this.
Alternatively, a speaker may be attached to 20, and an onomatopoeic sound may be sounded when the 3D polygon 10 jumps out onto the reference plane, or the 3D polygon 10 may then speak something.

The game device 100 of this embodiment is disclosed in Japanese Patent Application Laid-Open Nos. 10-123453 and 9-304727,
JP-A-9-304730, JP-A-9-21374
JP-A-8-160348, JP-A-8-94960
JP-A-7-325265, JP-A-7-27071
4 and JP-A-7-67055.

In this embodiment, the case where the two-dimensional imaging device of the interline transfer system is used for the panning CCD 23 has been described. However, the present invention is not limited to this, and the case where the two-dimensional imaging device of the frame transfer system is used. Even if there is, the same effect can be obtained.

[0103]

As described above, according to the virtual image three-dimensional composition device of the present invention, an object to be composed at a position that includes a virtual body in a virtual space, and is separated from a reference plane in a real space. Position detection means for detecting the position of the object in the real space with respect to the object at the position, and performing image processing such that the image of the virtual body changes in the virtual space based on the position information of the object Things.

With this configuration, from the position change of the object in the real space by the position detecting means, the displacement information such as the virtual approach, contact or press of the object in the virtual space can be easily obtained with a small amount of calculation. can do. Therefore, the virtual image three-dimensional composition device of the present invention can be sufficiently applied to a virtual character breeding game device and the like.

Further, according to the virtual image three-dimensional synthesis method of the present invention, at least an object to be synthesized at a position including a virtual body in a virtual space and at a position distant from a reference plane in a real space. Regarding an object, the position of the object is detected in a real space, and then image processing is performed so that the image of the virtual body changes based on the position information of the object.

With this configuration, it is possible to easily obtain the displacement information such as the virtual approach, contact, or press of the object in the virtual space from the position change of the object in the real space with a small amount of calculation. Therefore, the calculation load on the image processing unit can be reduced as compared with the conventional method, and the cost can be reduced when a virtual character breeding game device or the like is configured by applying the virtual image three-dimensional synthesis method of the present invention. .

Further, according to the game apparatus of the present invention, since the above-described virtual image three-dimensional synthesizing apparatus is applied, the object in the virtual space can be easily obtained from the position information of the object by the position detecting means with a small amount of calculation. Virtual contact, contact or pressing of the user can be recognized.

Therefore, as if a TV program character or the like jumps out to the position to which the reference plane in the real space belongs, and a character or the like placed on one palm of the observer is stroked with the other hand, You can make the character laugh, cry, or offend. This makes it possible to easily configure a virtual character breeding game device and the like, and to provide it at a low price.

Further, according to the recording medium of the present invention, an algorithm for executing the above-described virtual image three-dimensional composition method is stored. Can move the character image. Therefore, the above-described virtual character breeding game device and the like can be configured with good reproducibility.

The present invention is extremely suitable when applied to the production of a virtual character breeding game device for breeding TV program characters and the like with palms in a virtual space.

[Brief description of the drawings]

FIG. 1 is a perspective view showing a configuration example of a game device 100 to which a virtual image three-dimensional composition device as an embodiment according to the present invention is applied.

FIG. 2 is a perspective view showing a configuration example of a reference surface setting bracelet 1 used in the first embodiment.

FIG. 3 is a conceptual diagram illustrating a configuration example of a special glasstron 2 used in the game apparatus 100 as viewed from the front.

FIG. 4 is a conceptual diagram illustrating a configuration example of another special glasstron 20 used in the game apparatus 100 as viewed from the front.

FIG. 5 is a plan view showing an example of the internal configuration of the panning CCD device 23 of the special glasstron 2.

FIG. 6 is a cross-sectional view showing a configuration example of an optical system of the panning CCD device 23.

7A is a front view illustrating a configuration example of a baffle 30A as an object 30, and FIG. 7B is a front view illustrating a configuration example of another baffle 30B.

FIG. 8 is a conceptual diagram showing an example of the internal configuration of the clasp 30A.

FIG. 9 is a front view showing a configuration example of a data glove 30C as an object 30.

FIG. 10 is a front view showing a configuration example of a force sense presentation glove 30D as an object 30.

FIG. 11 is a diagram showing an example of a circuit block of the game device 100.

FIG. 12 is an image diagram showing an example of a normal image of the baffle 30A on the plate portion 11 forming the reference plane.

FIG. 13 is a schematic view showing an example of calculating the position coordinates of the reference plane and the buzzer 30A.

FIG. 14 is a conceptual diagram showing an example of display information stored in the E ² PROM 48.

FIG. 15 is a flowchart of a main routine showing an operation example (No. 1) of the game apparatus 100.

FIG. 16 is a flowchart of a subroutine showing an operation example (part 2) of the game apparatus 100.

17A is an example of a real image of the plate unit 11 in the real space, and FIG. 17B is an image diagram illustrating an example of the synthesis of the 3D polygon 10 on the reference plane in the virtual space.

FIG. 18 shows a reference plane setting 2 used in the second embodiment.
FIG. 4 is a plan view illustrating an example of a dimensional barcode 50.

FIG. 19 is a perspective view showing a synthesis example of the 3D polygon 10 on the two-dimensional barcode 50.

[Explanation of symbols]

1. Bracelet for setting reference surface (surface recognition means),
2, 20 special glasstron, 3 image processing device, 4 position detection mechanism (position detection means), 10
・ 3D polygon, 23: Panning CCD device, 24
... Display means, 25 ... CCD imaging device, 26 ...
LCD for right eye display (first image display element), 27
..LCD for displaying left eye (second image display element), 30
A, 30B: Jig (object), 30C: Data glove, 30D: Force glove, 32
..Vertical transfer unit (charge transfer unit), 33... Horizontal transfer unit, 50... Two-dimensional barcode, 100.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 17/40 H04N 5/272 H04N 5/272 13/02 13/02 13/04 13/04 A63F 9 / 22 B // A63F 13/00 G06F 15/62 350K

Claims (15)

[Claims] An apparatus for stereoscopically combining an image of a virtual body with an external world image to which an observer belongs, a plane recognition means for recognizing an arbitrary reference plane in a real space to which the observer belongs, and the plane Synthesizing means for synthesizing an image of the virtual body at a position to which the reference plane in the virtual space belongs so that the virtual body exists at a position to which the reference plane in the real space belongs, which is recognized by the recognition means. Position detecting means for detecting position information of the object synthesized on a position including the virtual body in the virtual space, the position information being located at a position distant from a reference plane in the real space; A virtual image three-dimensional composition apparatus, wherein image processing is performed such that an image of the virtual body changes in a virtual space based on position information of the object obtained from a detection unit. 2. A method of stereoscopically combining an image of a virtual body with an external image to which an observer belongs, wherein an arbitrary reference plane is set in a real space to which the observer belongs, and wherein the set reference plane is set. And synthesizes an image of the virtual body on the reference plane of the virtual space so that the virtual body exists at the position to which the reference plane in the real space belongs, at least a virtual body on the virtual space. Detecting the position information of the object located at a position distant from the reference plane in the real space on the virtual space based on the detected position information of the object. Wherein the image processing is performed so that the image of the virtual body changes. 3. An apparatus for stereoscopically combining an arbitrary character image with an external world image to which an observer belongs, a plane recognizing means for recognizing an arbitrary reference plane in a real space to which the observer belongs; Synthesizing means for synthesizing an image of the character on the reference plane of the virtual space as if the character is present at the position to which the reference plane in the real space belongs, which is recognized by the recognition means; The position of the object attached to a part of the body part of the observer or the object held by the observer combined with the position including the character, and located at a position away from the reference plane in the real space. Position detecting means for detecting information, wherein image processing is performed so that a character image in the virtual space changes based on position information of the object obtained from the position detecting means. Game device for the butterflies. 4. A case in which position detection means for detecting a position of the object is provided, wherein the position detection means is attached to an arbitrary position of the object and blinks at least in a different blinking pattern. It has two or more light sources, imaging means for imaging the light source so as to flow in a predetermined imaging direction, and arithmetic means for processing a luminance signal by the imaging means to obtain a position with respect to the reference plane. The game device according to claim 3, wherein 5. When the position detecting means is provided, an image processing means for performing image processing of the position information by the position detecting means is provided, and the image processing means is configured to execute the virtual processing on the reference position information acquired in advance. 4. The game according to claim 3, further comprising a memory storing display information for moving a body image, wherein the display information is read from the memory based on the position information detected by the position detecting means. apparatus. 6. The object provided with the two or more light sources is a glove worn on a part of a body part of the observer or a jig held by the observer. Game equipment. 7. A case in which a memory is provided in said image processing means, wherein at least first to third reference position information is stored in said memory, and a position included in said first reference position information. When information is detected, display information that makes the virtual body cry is read, and when position information included in the second reference position information is detected, a display that makes the virtual body laugh. 6. The game according to claim 5, wherein information is read, and when position information included in the third reference position information is detected, display information that makes the virtual body angry is read. apparatus. 8. The surface recognition means is attached to a position where an image of a virtual body is to be synthesized.
At least three or more light sources that blink so as to have different blink patterns, imaging means for imaging the light source so as to flow in a predetermined flowing imaging direction, and image processing of a luminance signal by the imaging means for the light source. 4. The game apparatus according to claim 3, further comprising: calculating means for obtaining a position of a point, and thereafter connecting the positions of the three light sources to obtain the reference plane. 9. The image pickup means uses a two-dimensional image pickup device having a plurality of photoelectric conversion elements constituting each pixel, and transfers a signal charge obtained from the photoelectric conversion element in a predetermined direction. 9. The game device according to claim 8, wherein the signal charge is read from the photoelectric conversion element at least a plurality of times during the same field period. 10. The apparatus according to claim 1, wherein said imaging unit and said calculation unit are provided, wherein said calculation unit is configured to:
By converting into a spatial arrangement pattern forming an XY plane including three luminescent points, scanning the arrangement pattern to obtain at least the position coordinates of three luminescent points, and connecting the position coordinates of the three points 9. The game device according to claim 8, wherein the reference plane is recognized. 11. The moving means, when the moving image pickup direction is an Y axis on an arrangement pattern including three bright points and an axis orthogonal to the Y axis is an X axis, The luminance signal values are added in the directions and plotted on the X-axis. The position where the luminance signal value plotted on the X-axis is maximum is detected to obtain three X-coordinate values. When scanning in the X-axis direction in (1), among the plurality of bright points arranged in the panning imaging direction, the position of the bright point that first emits light is obtained as a Y coordinate value corresponding to the X coordinate value. Item 10. The game device according to item 8. 12. The surface recognition means is attached to a position where an image of a virtual body is to be synthesized.
At least an n-row x n-column black-and-white matrix printed in black on a white background, a two-dimensional matrix code including a black frame having the same thickness as the black-and-white matrix, imaging means for imaging the two-dimensional matrix code, and imaging 4. The game apparatus according to claim 3, further comprising: an arithmetic unit for performing image processing on the luminance signal by the unit to obtain a reference plane from the two-dimensional matrix code. 13. An image capturing means for capturing an external image to which an observer belongs, and a first image for displaying one of a stereo image obtained by combining the external image by the image capturing means and an image of a virtual object prepared in advance. And a second image display element for displaying the other of the stereo images. The head mounted display is mounted on a face or a head of an observer, and the first 4. The game apparatus according to claim 3, wherein the stereo image by the image display element and the stereo image by the second image display element are superimposed and guided to an observer's eyeball. 14. A liquid crystal shutter for opening and closing incident light to capture an external image to which an observer belongs, an image display element for displaying an image of a virtual body to be combined with the external image, An image of the virtual body by the image display element, and a head mounted display having an optical unit that guides an external image to which the observer who has passed through the liquid crystal shutter belongs to the observer's eyeball, wherein the head mount display includes: When mounted on the face or head of the observer and the liquid crystal shutter is opened, the image of the virtual body by the image display element is superimposed on the external image in real space to which the observer who has passed through the liquid crystal shutter belongs. 4. The game device according to claim 3, wherein the game device is guided to an eyeball of an observer. 15. A recording medium storing an algorithm for stereoscopically combining an image of a virtual body with an external image to which an observer belongs, wherein the recording medium has an arbitrary reference in a real space to which the observer belongs. Setting a plane, imaging the set reference plane and synthesizing it in the virtual space, and superimposing and synthesizing the image of the virtual body on the virtual space, at least including the virtual body in the virtual space A virtual object in the virtual space based on position information of the detected object, wherein the virtual object in the virtual space is detected based on position information of the detected object. A recording medium storing an algorithm for performing image processing so that the image changes.