JP7304264B2 - 3D image display system - Google Patents

Description

The present invention relates to a stereoscopic image display system that forms a viewing zone for each eye of a person.

Integral photography (IP) methods have been studied since they do not require special spectacles and enable stereoscopic viewing with the naked eye. As shown in FIGS. 12(a) and 12(b), a conventional IP stereoscopic image display device 100 includes a direct-view display 110 such as liquid crystal or organic EL, and a lens array 120 placed in front of it. The lens array 120 is composed of element lenses 121 arranged two-dimensionally. The direct-view display 110 displays an elemental image group E composed of elemental images e corresponding to the elemental lenses 121 . Then, the elemental images e are projected in space by the elemental lenses 121 to form a stereoscopic image.

In the IP system, a very large number of pixels (amount of information) are required in order to reproduce images from many viewpoints. There are three parameters that determine the quality of IP stereoscopic video: the number of stereoscopic video pixels (the number of element images), the viewing zone, and the spatial frequency characteristics (depth reproduction range). As described in Non-Patent Document 1, these three parameters have a trade-off relationship. In other words, by narrowing the "viewing zone", it is possible to improve the "stereoscopic image pixel count" and the "spatial frequency characteristics". On the other hand, when the "viewing zone" is narrowed, the display range of stereoscopic images is narrowed. As shown in FIGS. 13A and 13B, in order to narrow the viewing zone V, an element lens 121 having a long focal length f may be used.

In the conventional IP method, if the direction of the viewing zone V is controlled by the viewing zone tracking technology, stereoscopic images can be displayed in a wide range even if a lens array having a long focal length f is used (Non-Patent Document 2). In this case, as shown in FIG. 14, by translating the display position of the elemental image group E in the direction opposite to the viewer 9, direction control of the viewing zone V can be realized (reduction/enlargement is also possible). In FIG. 14(a), since the viewer 9 has moved upward, the display position of the elemental image group E is moved downward, thereby moving the visual field V indicated by dots upward. In addition, in FIG. 14B, since the viewer 9 has moved downward, the viewing zone V is moved downward by moving the display position of the elemental image group E upward. In this way, in the IP system, the "three-dimensional image pixel count" and "spatial frequency characteristics" can be improved, and the viewing zone V can be widened.

In the conventional visual field tracking technology, as shown in FIG. 15, a virtual display 210, a virtual lens array 220, a virtual camera array 230 consisting of a plurality of virtual cameras 231, and a three-dimensional Place the model Obj. In the viewing zone tracking technique, pupil detection is performed and the position of the virtual camera array 230 is set so that both eyes are included in the viewing zone. Furthermore, in the viewing area tracking technology, the virtual camera array 230 captures the multi-viewpoint image group M of the three-dimensional model Obj in various directions, and converts the captured multi-viewpoint image group M into the element image group E. This multi-viewpoint image group M is composed of images of different viewpoints (multi-viewpoint images m).

H. Hoshino, F. Okano, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A 15(8), 2059-2065 (1998) Okaichi, Sasaki, Watanabe, Hisatomi, Kawakita, “Viewpoint-tracking integral 3D image display using 8K display”, 2018 Institute of Image Information and Television Engineers Winter Conference Preprints

In the prior art described above, as shown in FIG. 16(a), since the visual field V of a size that includes the viewer's 9 eyes (the left eye 90 _L and the right eye 90 _R ) is formed, the left eye 90 _L and the right eye _90R , the visual field V becomes redundant, and the utilization efficiency of pixel information is lowered. Therefore, as shown in FIG. 16(b), if a viewing zone V ( _VL , _VR ) is formed according to the size of one eye 90 ( _90L , _90R ) and time-divisionally displayed, the pixel information Utilization efficiency is increased, and high-efficiency and high-quality stereoscopic video viewing becomes possible. The viewing zone V _L is the viewing zone corresponding to the left eye _90L , and the viewing zone V _R is the viewing zone corresponding to the right eye _90R .

An object of the present invention is to provide a stereoscopic image display system with high utilization efficiency of pixel information.

In view of the above-described problems, a stereoscopic image display system according to the present invention generates a group of elemental images for each eye of a person from a group of multi-viewpoint images obtained by photographing a virtual space in which a 3D model is arranged with a virtual camera array. and a stereoscopic image display device that has a parallel optical system and displays a group of element images generated by the stereoscopic image generation device in a time division manner by forming a viewing zone for each eye. The stereoscopic image generation device includes virtual camera array movement means, multi-viewpoint image group photographing means, elemental image group generation means, and time-division control means.

According to such a stereoscopic image generating apparatus, the virtual camera array moving means receives pupil position information indicating the position of one eye, and moves the position of the virtual camera array based on the inputted pupil position information. That is, the virtual camera array moving means moves the position of the virtual camera array considering the position of one eye.

The multi-viewpoint image group shooting means shoots the multi-viewpoint image group with the virtual camera array moved by the virtual camera array moving means.
The elemental image group generating means generates an elemental image group from the multi-viewpoint image group.
The time-division control means receives pupil number information indicating the number of one eye, and controls the virtual camera array moving means, the multi-viewpoint image group photographing means, and the elements so as to perform time-division display based on the inputted pupil number information. It controls the image group generating means.
As described above, in the stereoscopic image display system, a viewing zone is formed for each eye by time-division display, so that the utilization efficiency of pixel information can be improved.

Further, in view of the above-mentioned problems, a stereoscopic image display system according to the present invention provides a group of element images for each eye of a person from a group of multi-viewpoint images obtained by photographing a virtual space in which a three-dimensional model is arranged with a virtual camera array. , a stereoscopic image display device that time-divisionally displays the group of element images generated by the stereoscopic image generation device for each eye, and a stereoscopic image display device that is worn on the head of a person and displays the group of elemental images in a time-division manner. A stereoscopic image display system comprising twin-lens stereoscopic glasses having shutters that open and close in synchronism, wherein a stereoscopic image generation device includes virtual camera array moving means, multi-viewpoint image group photographing means, and elemental image group generation means. , shutter control means, and time-sharing control means, wherein the stereoscopic image display device comprises elemental image group display means for displaying the elemental image group generated by the stereoscopic image generation device, and elemental lenses arranged in a two-dimensional manner. and a lens array arranged so that the elemental image group display means is positioned at the focal position of each elemental lens.

According to such a stereoscopic image generating apparatus, the virtual camera array moving means receives pupil position information indicating the position of one eye, and moves the position of the virtual camera array based on the inputted pupil position information. That is, the virtual camera array moving means moves the position of the virtual camera array considering the position of one eye.

The multi-viewpoint image group shooting means shoots the multi-viewpoint image group with the virtual camera array moved by the virtual camera array moving means.
The elemental image group generating means generates an elemental image group from the multi-viewpoint image group.
The shutter control means opens the shutter corresponding to one eye of the pupil position information and controls the opening and closing of the shutters so as to close the other shutters.
The time-division control means receives pupil number information indicating the number of one eye, and controls the virtual camera array moving means, the multi-viewpoint image group photographing means, and the elements so as to perform time-division display based on the inputted pupil number information. It controls the image group generation means and the shutter control means.
As described above, in the stereoscopic image display system, a viewing zone is formed for each eye by time-division display, so that the utilization efficiency of pixel information can be improved.

According to the present invention, it is possible to improve the utilization efficiency of pixel information.

1 is a block diagram showing the configuration of a stereoscopic image display system according to a first embodiment; FIG. It is an explanatory view explaining crosstalk in the conventional IP stereoscopic video display device. FIG. 4 is an explanatory diagram for explaining the design of the stereoscopic image display device and the mask image in the first embodiment; FIG. 4 is an explanatory diagram illustrating a method of generating an elemental image group in the first embodiment; (a) and (b) are explanatory diagrams for explaining viewing zone tracking in the first embodiment. FIG. 4 is an explanatory diagram for explaining time-division display of a group of elemental images in the first embodiment; 4 is a flow chart showing the operation of the stereoscopic image display system according to the first embodiment; FIG. 11 is a block diagram showing the configuration of a stereoscopic image display system according to a second embodiment; FIG. FIG. 11 is an explanatory diagram for explaining viewing zone tracking by time-division display of a group of elemental images in the second embodiment; FIG. 11 is a block diagram showing the configuration of a stereoscopic image display system according to a third embodiment; FIG. FIG. 12 is a block diagram showing the configuration of a stereoscopic image display system according to a fourth embodiment; FIG. (a) and (b) are explanatory diagrams for explaining a conventional IP stereoscopic video display device. (a) and (b) are explanatory diagrams for explaining the relationship between the viewing zone and the focal length of the element lens in the prior art. (a) and (b) are explanatory diagrams for explaining a conventional viewing zone tracking technique. FIG. 10 is an explanatory diagram for explaining a conventional viewing zone tracking technique; (a) is an explanatory diagram for explaining a conventional viewing zone corresponding to both eyes, and (b) is an explanatory diagram for explaining a viewing zone corresponding to one eye according to the present invention.

(First embodiment)
Hereinafter, each embodiment of the present invention will be described in detail with appropriate reference to the drawings. In addition, in each embodiment, the same code|symbol was attached|subjected to the same means, and description was abbreviate|omitted.

[Overview of stereoscopic image display system]
An overview of a stereoscopic image display system 1 according to the first embodiment will be described with reference to FIG.
As shown in FIG. 1, a stereoscopic image display system 1 displays an IP stereoscopic image (element image group E), and includes a pupil position detection device 2, a stereoscopic image display device 3, and a stereoscopic image generation device. 4.

In the stereoscopic image display system 1, a stereoscopic image display device 3 in which crosstalk does not occur is constructed. As shown in FIG. 2, crosstalk means that the light from the direct-view display 110 is diffused, and when an elemental image e is viewed through the lens array 120, an elemental image e adjacent to the elemental image e can be seen. It is a phenomenon. In the conventional IP stereoscopic video display device 100, this crosstalk appears as side lobes SL outside the main lobe (central viewing zone) ML. Therefore, in the stereoscopic image display system 1, the stereoscopic image display device 3 is designed so that crosstalk that disturbs viewing does not occur.

Further, in the stereoscopic image display system 1, the element lenses 33 of the stereoscopic image display device 3 are arranged so as to form a narrow viewing zone V (V _L , V _R ) capable of including one eye 90 (90 _L , 90 _R ). Design the pitch and focal length f of . As described above, there is a trade-off relationship between the three parameters of "viewing area", "stereoscopic image pixel count", and "spatial frequency characteristics". Therefore, by forming a narrow "viewing zone", it is possible to improve the "stereoscopic image pixel count" and the "spatial frequency characteristics".
In the stereoscopic image display system 1, the position of one eye 90 is detected by the pupil position detection device 2, and the element image group E corresponding to the position is alternately displayed by the one eye 90 in a time division manner.

In this way, the stereoscopic image display system 1 forms a narrow viewing zone V according to the size of one eye 90 and The time-division display of the element image group E according to the number of is applied. Therefore, the configuration of the stereoscopic image display system 1 will be described, focusing on the difference from the conventional viewing zone tracking technology.

[Configuration of stereoscopic image display system]
As shown in FIG. 1, the pupil position detection device 2 detects a single eye 90 of a viewer 9, and sets pupil position information indicating the position of the detected single eye 90 and pupil number information indicating the number of the single eye 90. is generated. For example, the pupil position detection device 2 includes a photographing device (for example, depth camera, eye tracker, web camera) for photographing the face image of the viewer 9 . The pupil position detection device 2 is preferably arranged above the stereoscopic image display device 3 so as to face the viewer 9 . Then, the pupil position detection device 2 detects the positions of the right eye 90 R and the left eye 90 L of each viewer 9 and the positions of the right eye 90 _R and the left eye 90 _L of each viewer 9 from the face image photographed by the photographing device, using pupil detection technology (for example, OpenCV). The total number of eyes 90 _R and left eyes 90 _L are detected in real time to generate pupil position information and pupil number information. After that, the pupil position detection device 2 outputs the generated pupil position information and pupil number information to the stereoscopic image generation device 4 .

The stereoscopic image display device 3 has a parallel optical system and time-divisionally displays the elemental image group E generated by the stereoscopic image generation device 4 for each eye 90 of the viewer 9 .
The stereoscopic image generation device 4 generates an elemental image group E for each eye 90 of the viewer 9 from the multi-viewpoint image group M captured by the virtual camera array 230 (FIG. 15). Then, the stereoscopic image generation device 4 outputs the generated elemental image group E to the stereoscopic image display device 3 . Details of the stereoscopic image generation device 4 will be described later.

<Stereoscopic image display device>
As shown in FIG. 1, the stereoscopic image display device 3 includes a projector (projection means) 30, a collimating lens 31 as a parallel light optical system, a lens array 32, and an LCD panel (elemental image group display means) 34. Prepare.

The projector 30 projects the mask image input from the stereoscopic image generation device 4 . This projector 30 is used as a backlight for an LCD panel 34, which projects light according to the non-masked areas (FIG. 3) of the masked image. For example, the projector 30 is a general one having a lens shift function and the like.

The collimating lens 31 emits the light of the mask image projected from the projector 30 as parallel light. For example, the collimating lens 31 is a biconvex lens arranged so that the projector 30 is positioned at its focal position.

The lens array 32 has element lenses 33 arranged two-dimensionally, and forms a point light source group by condensing the parallel light emitted from the collimating lens 31 at the focal position of each element lens 33. is. Here, the focal position of each element lens 33 is the focal point. For example, the lens array 32 has minute biconvex lenses as element lenses 33 and is arranged behind the collimator lens 31 .

The LCD panel 34 displays an elemental image group E generated by the stereoscopic image generating device 4 by a point light source group formed by the lens array 32 . For example, the LCD panel 34 is a general liquid crystal display (LCD) arranged behind the lens array 32 .

<<Design of stereoscopic image display device, mask image>>
The design of the stereoscopic image display device 3 and the mask image will be described in detail with reference to FIG.
In FIG. 3(a), a three-dimensional coordinate system with the focal point F of a certain element lens 33 as the origin (0, 0, 0), the horizontal direction as the x-axis, the vertical direction as the y-axis, and the depth direction as the z-axis. will explain. Let f be the focal length of the element lens 33, and w be the size of the element image e. Let l be the distance from the focal point F of the element lens 33 to the element image e. Let L be the distance from the focal point F of the element lens 33 to the viewpoint position. Note that the viewpoint position is the position of the eye 90 in the z-axis direction indicated by the pupil position information.
In FIG. 3B, the horizontal size of the lens element 33 is _ph , and the vertical size of the lens element 33 is _pv .

As shown in FIG. 3( a ), in the stereoscopic image display device 3 , the width of the light spread from the focal point F is the same as the size w of the elemental image e displayed on the LCD panel 34 . Here, the size of the viewing area V at the viewpoint position is represented by a value obtained by multiplying the size w of the element image e by the distance L and dividing by the distance l, as shown in the following equation (1). Therefore, in the stereoscopic image display device 3, the size of the viewing zone V obtained by Equation (1) is equal to or greater than the preset size α of the single eye 90 and within a certain range of the size α of the single eye 90. make it
w × L / l ... formula (1)

Further, as shown in FIG. 3B, the mask image J has a non-mask area (white area) _JW from which light is emitted and a mask area (black area) _JB from which light is not emitted. In addition, in FIG. 3B, the mask region _JB is illustrated by hatching. Here, the size β of the non-masked region _JW is obtained by multiplying the size w of the elemental image e by the focal length f of the elemental lens 33 and dividing by the distance l, as shown in the following equation (2). . Therefore, the size β of the non-masked area _JW may be set in advance according to equation (2), and the mask image J having the non-masked area _JW may be generated.
w×f/l...Equation (2)

Note that the mask image J in FIG. 3 represents one corresponding to one element lens 33 . Therefore, the mask image generating means 41 described later generates a two-dimensional arrangement of the mask images J in FIG. 3 according to the number of the element lenses 33 . Also, in FIG. 3, the horizontal direction has been described, but the same applies to the vertical direction.

<Stereoscopic image generation device>
Returning to FIG. 1, the stereoscopic image generation device 4 will be described.
As shown in FIG. 1, the stereoscopic image generation device 4 includes time-division control means 40, mask image generation means 41, virtual camera array initialization means 42, virtual camera array movement means 43, and three-dimensional model setting means. 44 , multi-viewpoint image group photographing means 45 , and elemental image group generating means 46 .

The time-division control means 40 receives pupil number information from the pupil position detection device 2, and controls the mask image generation means 41 and the virtual camera array movement means 43 so as to perform time-division display based on the inputted pupil number information. It controls the multi-viewpoint image group photographing means 45 and the elemental image group generating means 46 (hereinafter referred to as the virtual camera array moving means 43, etc.). In FIG. 1, the time-division control target of the time-division control means 40 is surrounded by a dashed line.

That is, the time-division control means 40 synchronously controls the virtual camera array moving means 43 and the like in accordance with the display timing of the one eye 90 . Specifically, the time-division control means 40 equally divides a unit time (for example, one second) by the number of the eyes 90 indicated by the pupil number information, and obtains the display timing of each eye 90 . At the display timing of each eye 90, the time-division control means 40 outputs the identification information of the eye 90 and the pupil position information of the eye 90 to the virtual camera array moving means 43 and the like. For example, the identification information for one eye 90 is a serial number assigned to left eye _90L and right eye _90R , respectively. As a result, the virtual camera array moving means 43 and the like can perform predetermined processing at the display timing of the single eye 90, and the elemental image group E can be displayed in a time division manner for each single eye 90. FIG. Details of viewing zone tracking by time-division display of the elemental image group E will be described later.

The mask image generation means 41 generates a mask image J (FIG. 3) having a non-mask area _JW based on the pupil position information input from the time-division control means 40 . Here, it is assumed that the size β of the non-masked area _JW is set in advance in the mask image generating means 41 as shown in the above equation (2). Also, the mask image generating means 41 forms the non-mask area _JW at a position corresponding to the pupil position information so that the projector 30 can project light in a direction corresponding to the position of the one eye 90 .
After that, the mask image generating means 41 outputs the generated mask image J to the projector 30 .

The virtual camera array initial setting means 42 initializes parameters relating to the virtual camera array 230 . For example, the parameters include the center position of the virtual camera array 230, the number of virtual cameras 231, their arrangement, their spacing, and their focal lengths. Specifically, the virtual camera array initialization means 42 sets the center position of the virtual camera array 230 to the position of one eye 90 based on the pupil position information input from the time division control means 40 . In addition, the creator of the stereoscopic image may set the number, arrangement, interval and focal length of the virtual cameras 231 in the virtual camera array initialization means 42 using operation means such as a mouse and keyboard (not shown). good.
After that, the virtual camera array initialization means 42 outputs the set parameters to the virtual camera array moving means 43 .

The virtual camera array moving means 43 receives the pupil position information from the time-division control means 40 and moves the position of the virtual camera array 230 based on the inputted pupil position information. That is, the virtual camera array moving means 43 moves the position of the virtual camera array 230 considering the movement of the one eye 90 .
After that, the virtual camera array moving means 43 outputs the position of the moved virtual camera array 230 to the multi-viewpoint image group photographing means 45 together with the parameters input from the virtual camera array initialization means 42 .

The three-dimensional model setting means 44 sets a three-dimensional model Obj to be placed in the virtual space. This three-dimensional model Obj is an object to be photographed by the virtual camera array 230, and is arbitrary like a woman wearing a hat (FIG. 15). For example, the creator of the stereoscopic image sets the three-dimensional model Obj to be arranged in the virtual space in the three-dimensional model setting means 44 using the operation means.
After that, the three-dimensional model setting means 44 outputs the set three-dimensional model Obj to the multi-viewpoint image group photographing means 45 .

The multi-viewpoint image group shooting means 45 shoots the multi-viewpoint image group M with the virtual camera array 230 moved by the virtual camera array moving means 43 . Specifically, the multi-viewpoint image group photographing means 45 photographs the virtual space in which the three-dimensional model Obj is arranged with the virtual camera array 230 arranged according to the position of the one eye 90 . In this manner, the multi-viewpoint image group photographing means 45 generates (renders) the multi-viewpoint image group M. FIG.
After that, the multi-viewpoint image group photographing means 45 outputs the generated multi-viewpoint image group M to the elemental image group generation means 46 .

The elemental image group generating means 46 generates an elemental image group E from the multi-viewpoint image group M input from the multi-viewpoint image group photographing means 45 . The elemental image group generating means 46 can generate the elemental image group E from the multi-viewpoint image group M by any method. For example, the elemental image group generating means 46 extracts pixels at the same position from each multi-viewpoint image m, and generates an elemental image e composed of the extracted pixels.
After that, the element image group generating means 46 outputs the generated element image group E to the stereoscopic image display device 3 .

<Generation of elemental image group>
An example of generation of the elemental image group E will be described with reference to FIG.
As shown in FIG. 4A, for a multi-viewpoint image m for three viewpoints, namely, an image m ₁₁ at the upper left viewpoint, an image m ₂₁ at the right viewpoint of the image m ₁₁ , and an image m ₁₂ at the lower viewpoint of the image m ₁₁ . think. In order to simplify the explanation, in the image _m11 , the upper left pixel is marked with "● 1", the right pixel is marked with "● 2", and the lower pixel is marked with "● 3". Similarly, " _▴1 " to "▴3" are added to the image m21, and "▪1" to "▪3" are added to the image _m12 .

In this case, as shown in FIG _. 4(b), the elemental image group generating means 46 extracts upper left pixels ●1, ▲ ₁ , _■ 1, . Generate image _e11 . That is, the element image _e11 is composed of upper left pixels ●1, ▲1, ■1, . . . at the same position. Also, in the element image _e11 , the arrangement of the upper left pixels ● ₁ , ▲ ₁ , ■ ₁ , . In other words, in the element image _e11 , the pixel ●1 is arranged on the upper left, the pixel ▲1 is arranged on the right side of the pixel ●1, and the pixel ■1 is arranged below the pixel ●1.

Further, the element image group generating means ₄₆ generates an element image _e21 composed of right pixels 2, _{2 ,} 2 _, . . . Generate. Further, the element image group generating means ₄₆ generates element images composed of lower pixels ● ₃ , ▲ ₃ , _{■ 3} , . . . Generate image _e12 .

Here, the elemental image e is arranged at a position corresponding to the pixel position of each pixel forming the elemental image e. For example, the element image _e11 is composed of the upper left pixels ●1, ▲1, ■1, . . . Also, since the element image _e21 is composed of the right pixels ●2, ▲2, ■ ₂ , . Further, the element image _e12 is arranged below the element image _e11 because it is composed of the lower pixels ●3, ▲3, ■3, .

Depending on the configuration of the optical system of the stereoscopic image display device 3, an inverted stereoscopic image may be displayed. In this case, the elemental image group generating means 46 may reverse the position of each pixel vertically and horizontally to generate the elemental image group E so that a stereoscopic erect image is displayed.

[Viewing zone tracking by time-division display of elemental images]
5 and 6, viewing zone tracking by time-division display of the elemental image group E in the stereoscopic image display system 1 will be described. 5 and 6, illustration of the pupil position detection device 2 and the stereoscopic image generation device 4 is omitted in order to make the drawings easier to see.

As shown in FIG. 5A, the projector 30 receives the mask image J from the mask image generating means 41 and projects white light corresponding to the non-mask area _JW . The collimating lens 31 collimates the light of the mask image J projected by the projector 30 and emits it as parallel light. The lens array 32 converges parallel light on the focal points F of the element lenses 33 to form a point light source group. Since the light spread from the condensing point F has the same size as the elemental image e displayed on the LCD panel 34, the elemental image group E is stereoscopically displayed.

Here, the pupil position detection device 2 detects the position of one eye 90, and the stereoscopic image generation device 4 generates the mask image J and the element image group E according to the position. That is, in the stereoscopic image display system 1, as shown in FIG. 5(b), the directions of light beams forming the elemental image group E are controlled according to the position of the one eye 90, so that viewing zone tracking can be achieved. Furthermore, in the stereoscopic image display system 1, since parallel light is emitted, crosstalk does not occur and the elemental image group E is not reproduced outside the viewing zone V, and the elemental image group E is displayed only in one eye 90. can be displayed.

As shown in FIG. 6, in the stereoscopic image display system 1, the elemental image group E is displayed in a time division manner. Here, in the stereoscopic image display system 1, since the number of one eye 90 is two, the element image group E may be time-divisionally displayed at intervals of one frame. For example, the stereoscopic image display system 1 displays the elemental image group E corresponding to the right eye 90 _R in odd-numbered frames #1, #3, #5, #7, . 4, #6, #8, . . . display the elemental image group E corresponding to the left eye _90L .

Although FIG. 6 illustrates the case where the number of viewers 9 is one, the stereoscopic image display system 1 can also cope with the case where there are a plurality of viewers 9 . In this case, the stereoscopic image display system 1 may perform time-division display according to the number of all eyes 90 detected by the pupil position detection device 2 .
In addition, in FIG. 6, the light rays emitted from the projector 30 are omitted in order to make the drawing easier to see, and only the viewing zones V _L and V _R formed by the light rays are shown.

[Operation of stereoscopic image display system]
The operation of the stereoscopic image display system 1 will be described with reference to FIG.
As shown in FIG. 7, in step S1, the pupil position detection device 2 detects one eye 90 of the viewer 9, and indicates pupil position information indicating the position of the detected one eye 90 and the number of one eye 90. pupil number information.
In step S<b>2 , the virtual camera array initialization means 42 initializes parameters regarding the virtual camera array 230 . For example, the center position of the virtual camera array 230 is set to the position of one eye 90 based on the pupil position information generated in step S1.

In step S3, the mask image generating means 41 generates a mask image J having a non-mask area _JW based on the pupil position information generated in step S1. For example, the mask image generating means 41 generates the mask image J such that the non-mask area _JW has a predetermined size and a position corresponding to the pupil position information.
In step S4, the virtual camera array moving means 43 moves the position of the virtual camera array 230 based on the pupil position information generated in step S1. That is, the virtual camera array moving means 43 moves the position of the virtual camera array 230 in consideration of the movement of the single eye 90 detected in step S1.

At step S5, the three-dimensional model setting means 44 sets a three-dimensional model Obj to be arranged in the virtual space.
In step S6, the multi-viewpoint image group shooting means 45 shoots the three-dimensional model Obj with the virtual camera array 230 moved in step S4, and generates a multi-viewpoint image group M.
In step S7, the elemental image group generating means 46 generates an elemental image group E from the multi-viewpoint image group M generated in step S6. For example, the elemental image group generating means 46 extracts pixels at the same position from each multi-viewpoint image m, and generates an elemental image e composed of the extracted pixels.

In step S8, the time-division control means 40 determines whether or not the time-division display of the elemental image group E should be ended. For example, when the last frame of the element image group E is displayed, the time-division control means 40 determines to end the time-division display.
If the time-division display is not finished (No in step S8), the stereoscopic image display system 1 proceeds to the process of step S9.
In step S9, the time-division control means 40 outputs the identification information and the pupil position information of the one eye 90 to be displayed next time-divisionally to the virtual camera array moving means 43, etc., and returns to the processing of step S3.
When ending the time-division display (No in step S8), the stereoscopic image display system 1 ends the operation.

[Action/effect]
As described above, the stereoscopic image display system 1 forms a viewing zone V for each eye 90 by time-division display, so that the utilization efficiency of pixel information is increased, and the elemental image group E can be viewed with high efficiency and high quality. becomes possible. In addition, since the stereoscopic image display system 1 performs stereoscopic display by the IP method, eye convergence and accommodation during viewing match, and compared to the binocular stereoscopic method, viewing for a long time is less tiring. Furthermore, since the stereoscopic image display system 1 has a high light density and satisfies the super multi-view condition in which a plurality of parallax images are incident in a single eye, the focus adjustment of the naked eye functions appropriately, and even long-time viewing does not cause fatigue. . Furthermore, in the stereoscopic image display system 1, since the viewer 9 directly sees the light from the projector 30, a bright elemental image group E can be displayed.

(Second embodiment)
[Configuration of stereoscopic image display system]
With reference to FIG. 8, the configuration of a stereoscopic image display system 1B according to the second embodiment will be described with respect to the differences from the first embodiment.
As shown in FIG. 8, in a stereoscopic image display system 1B, the configuration of a stereoscopic image display device 3B is different from that of the first embodiment. and
Note that the pupil position detection device 2 is the same as that of the first embodiment, so the description is omitted.

<Stereoscopic image display device>
As shown in FIG. 8, the stereoscopic image display device 3B includes an LED array (light source) 35, a collimator lens 36 as a parallel light optical system, an LCD panel 37, and a lens array 38.

The LED array 35 emits light having directivity based on the pupil position information input from the stereoscopic image generation device 4B. Then, the LED array 35 lights the high-brightness LED according to the pupil position information. In other words, the LED array 35 controls the direction of the viewing zone V by changing the high-brightness LED to be lit based on the pupil position information.

The collimating lens 36 emits the light from the LED array 35 as parallel light. For example, the collimating lens 36 is a biconvex lens positioned so that the LED array 35 is positioned at its focal position.

The LCD panel 37 displays an elemental image group E generated by the stereoscopic image generating device 4B with parallel light from the collimating lens 36. FIG. For example, the LCD panel 37 is a general liquid crystal display arranged behind the collimating lens 36 .

The lens array 38 has element lenses 39 arranged two-dimensionally, and is arranged so that the LCD panel 37 is positioned at the focal position of each element lens 39 . For example, the lens array 38 has minute biconvex lenses as element lenses 39 and is arranged behind the LCD panel 37 .

<Stereoscopic image generation device>
As shown in FIG. 8, the stereoscopic image generation device 4B includes time-division control means 40B, virtual camera array initialization means 42, virtual camera array movement means 43, three-dimensional model setting means 44, and multi-viewpoint image groups. It has a photographing means 45 , an elemental image group generating means 46 and an LED array control means 47 .
Note that each means other than the time-division control means 40B and the LED array control means 47 are the same as those in the first embodiment, so description thereof will be omitted.

The time-division control means 40B receives pupil number information from the pupil position detection device 2, and controls the virtual camera array moving means 43 and the like so as to perform time-division display based on the inputted pupil number information. . Furthermore, the time-division control means 40B outputs the identification information of the one eye 90 and the pupil position information of the one eye 90 to the LED array control means 47 in accordance with the display timing of the one eye 90 . In other respects, the time-division control means 40B is the same as in the first embodiment, so further explanation is omitted.

The LED array control means 47 commands the position of each LED to be lit by the LED array 35 based on the pupil position information input from the time-division control means 40B. Specifically, the LED array control means 47 outputs a control signal to the LED array 35 so as to form a viewing zone in the direction of one eye 90 .

[Viewing zone tracking by time-division display of elemental images]
With reference to FIG. 9, viewing zone tracking by time division display of the elemental image group E in the stereoscopic image display system 1B will be described.
As shown in FIG. 9, the stereoscopic image display system 1B uses an LED array 35 as a directional backlight. A collimating lens 36 collimates the light from the LED array 35 and emits it to the LCD panel 37 . Parallel light from the LCD panel 37 passes through the lens array 38, so that the elemental image group E is stereoscopically displayed.

Here, the pupil position detection device 2 detects the position of one eye 90, and the stereoscopic image generation device 4B generates an elemental image group E corresponding to the position. Furthermore, since the LED array 35 lights a high-brightness LED according to the position of the one eye 90, visual field tracking can be achieved. Furthermore, in the stereoscopic image display system 1B, since parallel light is emitted, crosstalk does not occur and the elemental image group E is not reproduced outside the viewing zones V _L and V _R . Image group E can be displayed.

Also, in the stereoscopic image display system 1B, the elemental image group E is displayed in a time division manner. Here, in the stereoscopic image display system 1B, since the number of one eye 90 is two, the element image group E may be time-divisionally displayed at intervals of one frame. For example, the stereoscopic image display system 1B displays the elemental image group E corresponding to the right eye 90 _R in odd-numbered frames #1, #3, #5, #7, . 4, #6, #8, . . . display the elemental image group E corresponding to the left eye _90L .

[Action/effect]
As described above, in the stereoscopic image display system 1B, as in the first embodiment, the utilization efficiency of pixel information is increased, and the elemental image group E can be viewed with high efficiency and high quality. In addition, the stereoscopic image display system 1B is less tiring even after viewing for a long time, as in the first embodiment. Furthermore, the stereoscopic image display system 1B can reduce the thickness of the stereoscopic image display device 3B.

(Third embodiment)
[Configuration of stereoscopic image display system]
With reference to FIG. 10, the configuration of a stereoscopic image display system 1C according to the third embodiment will be described with respect to the differences from the second embodiment.
In the stereoscopic image display system 1B (FIG. 8) according to the second embodiment, the focal length of the collimating lens 36 is long, so the size in the depth direction is large. Therefore, a stereoscopic image display system 1C according to the third embodiment has an array of collimating lenses 36b that form parallel light.

As shown in FIG. 10, the stereoscopic image display system 1C includes a pupil position detection device 2, a stereoscopic image display device 3C, and a stereoscopic image generation device 4B. Note that the pupil position detection device 2 and the stereoscopic image generation device 4B are the same as those in the second embodiment, so descriptions thereof will be omitted.

<Stereoscopic image display device>
As shown in FIG. 10, the stereoscopic image display device 3C includes an LED array 35, a collimator lens array 36B as a parallel light optical system, an LCD panel 37, and a lens array 38. Note that each unit other than the collimating lens array 36B is the same as in the second embodiment, and thus description thereof is omitted.

The collimating lens array 36B has collimating lenses 36b arranged two-dimensionally, and is arranged so that the LED array 35 is positioned at the focal position of each collimating lens 36b. For example, the collimating lens 36b is a biconvex lens with a shorter focal length than the collimating lens 36 of FIG. The collimating lens array 36B emits the light from the LED array 35 as parallel light.

[Action/effect]
As described above, in the stereoscopic image display system 1C, similarly to the second embodiment, the utilization efficiency of pixel information is increased, and the elemental image group E can be viewed with high efficiency and high quality. Further, the stereoscopic image display system 1C is less tiring even after long hours of viewing, as in the second embodiment. Furthermore, in the stereoscopic image display system 1C, the focal length of the collimating lens array 36B is short, so the stereoscopic image display device 3B can be significantly thinned.

(Fourth embodiment)
[Configuration of stereoscopic image display system]
With reference to FIG. 11, the configuration of a stereoscopic image display system 1D according to the fourth embodiment will be described with respect to the differences from the second embodiment.
A stereoscopic image display system 1D according to the fourth embodiment includes twin-lens stereoscopic glasses 5 in place of the parallel light optical system in order to prevent crosstalk.
As shown in FIG. 11, the stereoscopic image display system 1D includes a pupil position detection device 2, a stereoscopic image display device 3D, a stereoscopic image generation device 4D, and binocular stereoscopic glasses 5. FIG.
Note that the pupil position detection device 2 is the same as that of the first embodiment, so the description is omitted.

<Binocular Stereo Glasses>
Here, the stereoscopic image display device 3D and the stereoscopic image generation device 4D will be described after the twin-lens stereoscopic glasses 5 are described.
The twin-lens stereoscopic glasses 5 have shutters 50 (50 _L , 50 _R ) and are worn on the head of the viewer 9, like general stereoscopic glasses used in the twin-lens stereoscopic system. For example, the shutter 50 is a liquid crystal shutter that opens and closes in synchronization with the time-division display of the elemental image group E according to a control signal input from the shutter control means 48 . Here, the left shutter 50 _L corresponds to the left eye 90 _L of the viewer 9 and the right shutter 50 _R corresponds to the right eye 90 _R of the viewer 9 . Specifically, at the timing of displaying the elemental image group E corresponding to the left eye _90L , the left shutter _50L opens and the right shutter _50R closes. On the other hand, at the timing of displaying the elemental image group E corresponding to the right eye _90R , the right shutter _50R opens and the left shutter _50L closes.

Although only one pair of stereoscopic glasses 5 is illustrated in FIG. 11 , each viewer 9 may wear the stereoscopic glasses 5 when there are a plurality of viewers 9 . In this case, the stereoscopic image display system 1D may control the opening and closing of the shutters 50 of the twin-lens stereoscopic glasses 5 worn by the respective viewers 9 .

<Stereoscopic image display device>
The stereoscopic image display device 3D is similar to the general IP stereoscopic video display device 100 (FIG. 12), and includes a direct-view display 310 and a lens array 320 placed in front of it.

The direct-view display 310 displays the elemental image group E generated by the stereoscopic image generating device 4D. For example, the direct-view display 310 is a display such as liquid crystal or organic EL.

The lens array 320 has element lenses 321 (for example, minute biconvex lenses) arranged two-dimensionally, and is arranged so that the direct-view display 310 is positioned at the focal position of each element lens 321 . .

<Stereoscopic image generation device>
The stereoscopic image generation device 4D includes time-division control means 40D, virtual camera array initialization means 42, virtual camera array movement means 43, three-dimensional model setting means 44, multi-viewpoint image group photographing means 45, and element images. A group generation means 46 and a shutter control means 48 are provided.
Since the components other than the time-division control means 40D and the shutter control means 48 are the same as those of the first embodiment, description thereof is omitted.

The time-division control means 40D receives pupil number information from the pupil position detection device 2, and controls the virtual camera array moving means 43 and the like so as to perform time-division display based on the inputted pupil number information. . Furthermore, the time-division control means 40D outputs the identification information of the one eye 90 and the pupil position information of the one eye 90 to the shutter control means 48 in accordance with the display timing of the one eye 90 . Other points of the time-division control means 40D are the same as those of the first embodiment, so further explanation is omitted.

The shutter control means 48 controls the opening and closing of the shutters 50 so that the shutters 50 corresponding to the pupil position information input from the time-division control means 40D are opened and the other shutters 50 are closed. Here, the shutter control means 48 outputs a control signal for controlling the opening and closing of the shutters 50 to the binocular stereoscopic glasses 5 at the timing when the pupil position information is input from the time-division control means 40D. Specifically, the shutter control means 48 outputs a control signal for opening the left shutter _50L and closing the right shutter _50R at the timing of displaying the elemental image group E corresponding to the left eye _90L . On the other hand, the shutter control means 48 outputs a control signal for opening the right shutter _50R and closing the left shutter _50L at the timing of displaying the elemental image group E corresponding to the right eye _90R .

[Action/effect]
As described above, in the stereoscopic image display system 1D, similarly to the first embodiment, the utilization efficiency of pixel information is increased, and the elemental image group E can be viewed with high efficiency and high quality. Furthermore, the stereoscopic image display system 1D does not cause fatigue even after long hours of viewing, as in the first embodiment.

(Modification)
Although the respective embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described respective embodiments, and includes design changes and the like within the scope of the present invention.

In each of the above-described embodiments, the pupil positions of one person are detected, but the present invention is not limited to this. In other words, the pupil positions of a plurality of people are detected, and time-division display is performed according to the number of people whose pupil positions are detected.

In each of the above-described embodiments, the present invention is applied to the IP stereoscopic method having omnidirectional parallax, but the present invention is not limited to this. INDUSTRIAL APPLICABILITY The present invention can be applied to a light-reproducing stereoscopic system such as an IP stereoscopic system and a lenticular stereoscopic system. This lenticular stereoscopic system is a light-reproducing stereoscopic system having only horizontal parallax.

1, 1B-1D stereoscopic image display system 2 pupil position detection device 3, 3B-3D stereoscopic image display device 4, 4B, 4D stereoscopic image generation device 5 binocular stereoscopic glasses 9 viewer 30 projector (projecting means)
31 collimating lens (parallel light optical system)
32 lens array 33 element lens 34 LCD panel (element image group display means)
35 LED array (light source)
36, 36b collimating lens (parallel light optical system)
36B collimating lens array (parallel light optical system)
37 LCD panel 38 Lens array 39 Element lenses 40, 40B, 40D Time division control means 41 Mask image generation means 42 Virtual camera array initialization means 43 Virtual camera array movement means 44 Three-dimensional model setting means 45 Multi-viewpoint image group photographing means 46 Elemental image group generation means 47 LED array control means 48 Shutter control means 50, 50 _L , 50 _R shutter 90 One eye 90 _L left eye 90 _R right eye 100 IP stereoscopic image display device 110, 310 direct-view display 120, 320 lens array 121, 321 element lens 210 virtual display 220 virtual lens array 230 virtual camera array 231 virtual camera V, _VL , _VR viewing area

Claims (7)

A stereoscopic image generating device for generating a group of elemental images for each eye of a person from a group of multi-viewpoint images obtained by photographing a virtual space in which a three-dimensional model is arranged by a virtual camera array; A stereoscopic image display system that forms a viewing zone for each eye and time-divisionally displays a group of element images generated by a generating device, the stereoscopic image display system comprising:
The stereoscopic image generating device is
virtual camera array moving means for receiving pupil position information indicating the position of one eye and moving the position of the virtual camera array based on the inputted pupil position information;
multi-viewpoint image group photographing means for photographing the multi-viewpoint image group with the virtual camera array moved by the virtual camera array moving means;
elemental image group generating means for generating the elemental image group from the multi-viewpoint image group;
The virtual camera array moving means, the multi-viewpoint image group photographing means, and the element images are input so as to perform time division display based on the inputted pupil number information. a time-sharing control means for controlling the group generation means;
A stereoscopic image display system comprising: The stereoscopic image display device
projection means for projecting a mask image having a non-mask area from which light is emitted, based on the pupil position information;
a collimating lens that emits the light of the mask image projected from the projection means as parallel light;
a lens array that has element lenses arranged in a two-dimensional pattern and that forms a point light source group by condensing parallel light emitted from the collimating lens at a focal position of each element lens;
elemental image group display means for displaying an elemental image group generated by the stereoscopic image generation device by means of a point light source group formed by the lens array;
The stereoscopic image display system according to claim 1, characterized by comprising: The stereoscopic image generation device multiplies the size of the elemental image by the focal length of the elemental lens and divides the result by the distance from the focal position of the elemental lens to the elemental image. 3. The stereoscopic image display system according to claim 2, further comprising mask image generating means for generating the mask image having the non-masked area which is set in advance. The stereoscopic image display device
a light source that emits light having directivity based on the pupil position information;
a collimating lens arranged so that the light source is positioned at a focal position and emitting the light from the light source as parallel light;
Elemental image group display means for displaying an elemental image group generated by the stereoscopic image generation device with parallel light from the collimating lens;
a lens array having element lenses arranged two-dimensionally and arranged so that the element image group display means is positioned at a focal position of each element lens;
The stereoscopic image display system according to claim 1, characterized by comprising: The stereoscopic image display device
a light source that emits light having directivity based on the pupil position information;
a collimating lens array having collimating lenses arranged in a two-dimensional pattern, arranged so that the light source is positioned at the focal position of each collimating lens, and emitting light from the light source as parallel light;
Elemental image group display means for displaying an elemental image group generated by the stereoscopic image generation device with parallel light from the collimating lens array;
a lens array having element lenses arranged two-dimensionally and arranged so that the element image group display means is positioned at a focal position of each element lens;
The stereoscopic image display system according to claim 1, characterized by comprising: A stereoscopic image generating device for generating a group of elemental images for each eye of a person from a group of multi-viewpoint images obtained by photographing a virtual space in which a three-dimensional model is arranged by a virtual camera array, and elemental images generated by the stereoscopic image generating device. A stereoscopic image display device for time-divisionally displaying a group for each eye, and twin-lens stereoscopic glasses that are worn on the head of the person and have shutters that open and close in synchronization with the time-divisional display of the elemental image group. A stereoscopic image display system,
The stereoscopic image generating device is
virtual camera array moving means for receiving pupil position information indicating the position of one eye and moving the position of the virtual camera array based on the inputted pupil position information;
multi-viewpoint image group photographing means for photographing the multi-viewpoint image group with the virtual camera array moved by the virtual camera array moving means;
elemental image group generating means for generating the elemental image group from the multi-viewpoint image group;
shutter control means for controlling the opening and closing of the shutter so as to open the shutter corresponding to one eye of the pupil position information and to close the other shutter;
The virtual camera array moving means, the multi-viewpoint image group photographing means, and the element images are input so as to perform time division display based on the inputted pupil number information. a time-division control means for controlling the group generation means and the shutter control means;
The stereoscopic image display device
Elemental image group display means for displaying the elemental image group generated by the stereoscopic image generation device;
a lens array having element lenses arranged two-dimensionally and arranged so that the element image group display means is positioned at a focal position of each element lens;
A stereoscopic image display system comprising: In the stereoscopic image display device, a value obtained by multiplying the size of the elemental image by the distance from the focal position of the elemental lens to the one eye and dividing by the distance from the focal position of the elemental lens to the elemental image is set in advance. 7. The stereoscopic image display system according to any one of claims 2 to 6, wherein the stereoscopic image display system is within a certain range with respect to the size of the single eye.

Images