JP4660769B2 - Multi-view stereoscopic display device - Google Patents

Description

  The present invention relates to a multi-view stereoscopic display in which a plurality of persons can simultaneously view a stereoscopic image from a unique viewpoint, can be viewed without feeling the fatigue of eyes peculiar to stereoscopic vision, and has a simple structure.

  3D display devices can be displayed closer to the real thing, so in addition to display for display, they are used in computer games, simulators for training, display devices for remote operation of devices, etc. is doing. However, it is known that when a stereoscopic display device is used for a long time, fatigue different from that of a normal flat display device remains.

  The direction of the line of sight when staring at a certain point in space (convergence: movement that becomes a cross-eyed when looking at nearby objects), the amount of image shift (binocular parallax) seen in the left and right eyes, the thickness of the lens It is known that a solid is perceived by three (focal length). For example, a stereoscopic display using glasses uses a polarizing filter or a liquid crystal shutter to input different images to the left and right eyes, thereby realizing binocular parallax and a convergence angle. However, these three-dimensional display methods have a problem in that there is a physiological contradiction between convergence and focus adjustment, and the eye fatigue peculiar to stereoscopic vision is caused by the convergence angle between both eyes and the focus adjustment of each eye. It is said that this is largely due to the contradiction in congestion adjustment. Many methods have already been proposed to solve this problem.

  One of them is a ray reproduction type solution, and a typical example is a hologram. However, with holograms, the amount of information to be processed is enormous, so it is difficult at present to achieve this with an electronic display. Further, as a method for realizing electronically fine light beam reproduction, a super multi-view system is known (Non-Patent Documents 1 and 2). However, even with this method, the amount of data to be prepared is very large, and in order to realize this, the resolution (number of pixels) of the image must be sacrificed.

  A method of combining a cylinder lens and a high-frequency pattern has been proposed as a method for solving congestion adjustment contradiction while suppressing the amount of information to be processed (Non-Patent Document 3). Although this method can significantly reduce the increase in the number of necessary data, since the high frequency pattern is superimposed on the original image, the deterioration of the image quality due to this is inevitable.

  On the other hand, several methods for realizing a volume display by rotating an image at a high speed or overlapping a plurality of screens have been proposed (Non-Patent Documents 4, 5, and 6). In these schemes, the light source is placed at the corresponding depth, so there is no congestion adjustment contradiction. However, in the case of a volume display, there arises a problem that generally an occlusion or gloss between objects cannot be expressed. In order to realize volume display, naturally, volume information of the video space is required. Accurate three-dimensional measurement in real time in the real world is difficult, and it is generally difficult to display a real image on a volume display.

  As a technique for solving all of the above problems at the same time, the inventors of the present invention have proposed a stereoscopic video display method in which only the edge region is displayed on a volume display and the flat region (region other than the edge) is displayed in a stereo system. (Non-Patent Document 7). However, since this method displays a large number of monochrome liquid crystals side by side on the volume display, the apparatus must be somewhat expensive. In addition, there remains a problem of how to construct a multi-view stereoscopic display for displaying a flat area in a stereo manner in a compact and inexpensive manner.

Yoshihiro Yuki, Hiroshi Yoshikawa, Ikuo Honda, Super multi-view 3D display with focused light source array (FLA), Proceedings of 3D image conference 1996, pp.108-113, 1996. Yasuhiro Takagi, Displaying 3D Objects Combining with a 64 Eye 3D Color Display, 3D Image Conference 2002 Proceedings, pp. 85-88, 2002. Hideki Kakeya, Tsuyoshi Akutsu, A method for realizing 3D image presentation without congestion contradiction at a low cost, 3D Image Conference 2004 Proceedings, pp. 9-12, 2004. S. Suyama, M. Date, H. Takada, Three-dimensional Display System with Dual-Frequency Liquid-Crystal Varifocal Lens, Jpn. J. April. Phys., 39, pp. 480-484, 2000. Actuality System, Perspecta Spacial 3D, http://www.actuality-systems.com/ A. Sullivan, LightSpace Technologies, Inc., DepthCube solid state 3D volumetric display, Stereoscopic Display and Virtual Reality Systems XI, SPIE Vol.5291, pp. 279-284, 2004. R. Yasui, I Matsuda, H Kakeya, Combining volumetric edge display and multiview display for expression of natural 3D images, SPIE proceeding Volume 6055: Stereoscopic Displays and Virtual Reality Systems XIII, pp. 0Y1 ・ 0Y9, 2006. Takayoshi Ohkoshi, 3D image engineering, Asakura Shoten, 1991. T. Hattori, T. Igarashi, K. Shimamoto, A. Sawaki, T. Ishiguchi, H. Kobayashi, Advanced autostereoscopic display for G-7 pilot project, SPIE proceedings Volume 3639: Stereoscopic Displays and Virtual Reality Systems VI, pp. 66 -75,1999.

JP 2005-165236 A

  In the present invention, an inexpensive construction method of a multi-view stereoscopic display that eliminates the congestion adjustment contradiction is proposed, and a new optical system is proposed to make the multi-view stereoscopic display simple. As a method for resolving the contradiction of congestion adjustment, a method for displaying a multi-layered multi-viewpoint video and a method for displaying a multi-layered background edge over the multi-viewpoint video are proposed.

  According to the present invention, a multi-view three-dimensional display in which congestion adjustment contradiction is suppressed can be constructed at low cost.

  The present invention relates to a multi-view three-dimensional display system that can make the entire apparatus as simple as possible while making use of the above-described edge separation display method. In the following, first, the improvement of the multi-view stereoscopic display unit will be described, and then, as a method for solving the contradiction of congestion adjustment by a simple method, a multi-view stereoscopic image to be displayed is layered, and edge patterns are layered in layers. The presentation method will be described.

[Improved multi-view 3D display]
As a multi-view stereoscopic display method, the inventors of the present invention have already proposed multi-view FLOATS shown in FIG. 1 (Patent Document 1). Of these devices, the Fresnel lens close to the observer is for the purpose of generating an aerial image, and even if this is omitted, it functions as a stereoscopic display where the fly-eye lens is the focal point adjustment point. Furthermore, considering the optical system from which the fly-eye lens is removed, the combination of the LCD (liquid crystal display device) and the convex lens array is regarded as a projector array, and is also described in textbook level text (Non-Patent Document 8). Corresponds to an extended version of the classic stereo display system with a convex lens optical system.

  The reason why a fly-eye lens is necessary in the above case is that, as shown in FIG. 2, when an image is observed from an angle greatly deviated from the optical axis of the lens, image formation by the projector is not possible except at the center of the screen. It will be enough. If this is left unattended, the position of the image greatly fluctuates depending on the viewpoint position. For this reason, it is not possible to observe a stereo image that is localized without distortion. Therefore, in the multi-viewpoint FLOATS, a fly-eye lens having a sufficiently long focal length of each minute lens is inserted to diffuse light slightly. By this method, the image is localized on the fly-eye lens surface, and the image shake due to the movement of the viewpoint position is eliminated. However, this time, insufficient condensing on the fly-eye lens surface appears in the form of image blurring, resulting in a problem that the sharpness of the displayed stereoscopic image is lowered.

  As an optical system capable of observing a stable stereoscopic image even at a viewpoint away from the optical axis, in the present invention, the distance between the LCD unit and the lens array unit is the focal length of each lens constituting the lens array. We propose an optical system that is arranged so as to be substantially equal. In this case, since the light coming out of the lens array becomes almost parallel light, no image is formed as it is. Therefore, the power of the Fresnel lens in front is used for imaging. This Fresnel lens is originally intended to be distributed so that an image corresponding to each viewpoint can be observed only from each viewpoint, but it is an idea that this is also used for imaging. In this case, the real image is formed in the air in front of the Fresnel lens. Although it is the same as FLOATS in FIG. 1 in that the aerial image is generated, since the generation of the real image is only once, there is an advantage that the optical path length of the entire system is greatly shortened.

[Resolution of congestion adjustment contradiction]
The main reason that observers feel eye fatigue and sickness when viewing stereoscopic images is that there is a discrepancy between the vergence angle of the eyes based on binocular parallax and the focus adjustment that makes each eye clearly see the image. It is said that it has become. As a method for solving this, it is conceivable to use a plurality of video presentation surfaces instead of one and present images at a plurality of depths. By displaying the stereoscopic image to be presented on the presentation surface closest to the original depth, the congestion adjustment contradiction can be resolved. However, in order to practice this method, it is necessary to know from which depth the pixels come from all the pixels of the stereoscopic image to be displayed. Therefore, application in computer graphics is possible, but application in live-action video is difficult.

  Reduction of congestion adjustment contradiction in live action can be realized by the following method. When a human gazes at an object, it is considered that information on edges in the image is used as a clue for focus adjustment. The human tries to focus on the distance where the blur of the edge is the smallest. Utilizing this property, the inventors of the present invention have proposed a method of contradicting congestion adjustment by extracting only edge information in an image and displaying it on a volume display (Non-patent Document 7). Specifically, a smoothed image is used for presentation of binocular parallax and convergence, and an edge image is arranged so as to overlap with the presented stereoscopic image. In this method, it is necessary to know the depth of the pixel only for the edge portion, but if this is the case, real shooting can be realized with considerable accuracy by the stereo matching method.

  Furthermore, an approach for simplifying the entire system can be considered here. The method described above is a method in which edges included in a stereoscopic image are positively extracted and displayed. However, there are edges that are not originally intended in electronic displays. It is the edge of the image pitch. It cannot be denied that this may lead to some focus adjustment. In view of this, it is conceivable to conversely actively use this phenomenon and intentionally disperse and display a certain fine pattern such as an image pitch as a background at a plurality of depths.

  When this method is actually combined with a multi-view stereoscopic display, a transparent sheet on which a fine edge pattern is printed is inserted before or after the LCD corresponding to each viewpoint, or an aerial image is generated. By arranging it in the vicinity where an aerial image is formed, a layered background edge can be displayed superimposed on the multi-viewpoint image.

For this reason, the multi-view stereoscopic display device of the present invention includes a display surface for displaying pixels, and a plurality of condensing systems that emit light from the display surface and emit substantially parallel light rays. Condensation system array formed adjacent to each other along with a large-diameter condensing beam that is incident on parallel light beams from the above-described condensing system array and exits toward each of the eyes of the observer A plurality of display surfaces provided at positions before or after the display surface on a line connecting the system and the display surface and the observer, farther than the light collection system array as viewed from the observer ; Alternatively, on a line connecting the display surface and the observer, a plurality of displays provided between the large-diameter condensing system and the observer and at positions before or after the imaging surface of the large-diameter condensing system A surface. The large-diameter condensing system forms an image on an imaging plane between the large-diameter condensing system and the observer. However, the image forming surface here is a virtual two-dimensional surface extending from the image forming point of the image on the display surface. In addition, parallel rays that are substantially parallel do not form an image between the above-described condensing system array and the large-diameter condensing system, but are parallel within a range in which an image is formed between the large-diameter condensing system and the observer. Light rays.

  A convex lens may be sufficient as the element of said condensing system array, or a large aperture condensing system.

  The elements of the above-described light condensing system array or the large-diameter condensing system may be a condensing system in which a lens or a mirror is combined.

In addition, three-dimensional image information is displayed using the display surface and the plurality of display surfaces. An image corresponding to the depth position of the image displayed on the display surface is displayed on the plurality of display surfaces. The image on the display surface and the plurality of display surfaces further provided are all observers. 3D display can be performed by making it visible from.

Further, in the multi-viewpoint three-dimensional display device, on the line connecting the display surface and the observer, the distant from the light collection system array as viewed from the observer, and at a position before or after the display surface. Alternatively, on the line connecting the display surface and the observer, the display is located at a position between the large-diameter condensing system and the observer and before or after the imaging surface of the large-diameter condensing system. Provide multiple edge pattern display surfaces to display the edge pattern of the image displayed on the screen or a predetermined background edge pattern, and display the edge pattern on the edge pattern display surface corresponding to the depth position of the above image By providing a configuration in which all of the image on the display surface and the edge pattern on the edge pattern display surface are visible to the observer, the congestion adjustment is performed. Contradiction can be eliminated.

  As shown in FIG. 11, the present invention includes a display surface 1 for displaying pixels and a plurality of condensing systems that emit light from the display surface 1 and emit parallel rays that are substantially parallel to each other. And a collimated light system array 2 formed adjacent to each other along the line and parallel light beams substantially parallel from the light collecting system array 2 are incident and emitted toward the eyes of the observer 5 respectively. The large-diameter condensing system 3 is configured to form an image on the imaging plane 4 between the large-diameter condensing system 3 and the observer 5. However, the image forming surface here is a virtual two-dimensional surface extending from the image forming point of the image on the display surface. Further, a parallel light beam that is substantially parallel does not form an image between the above-described condensing system array 2 and the large-diameter condensing system 3, but forms an image between the large-diameter condensing system 3 and the observer 5. Parallel rays in range. The display surface 1 is controlled by a display controller 6.

  In the present invention, as shown in FIG. 12, a plurality of display surfaces, 7a and 7b in FIG. 12, are further provided in the vicinity of the display surface 1, and the display surface 1 and the plurality of display surfaces are provided. 3D image information is displayed, and an image corresponding to the depth position of the image displayed on the display surface 1 is displayed on the plurality of display surfaces. An image and a plurality of further provided display surfaces are arranged so that they can be seen by an observer. Here, (7a, 7b) is an example, and the plurality of display surfaces are not limited to these two surfaces.

  In addition, as shown in FIGS. 13 and 14, the present invention is located in the vicinity of the display surface 1 or in the vicinity of the imaging surface 4 at a position that moves back and forth on the line connecting the display surface 1 and the observer 5. A plurality of edge pattern display surfaces for displaying the edge pattern of the image displayed on the display surface 1, 11a and 11b in FIG. 13, or 12a and 12b in FIG. 14, are provided at the depth position of the image. The edge pattern is displayed on the corresponding edge pattern display surface, and is arranged so that the image on the display surface 1 and the edge pattern on the edge pattern display surface are all visible to the observer. The display surface 1 forms an image on the image forming surface 4, and the edge pattern display surface, 11a and 11b in FIG. 13, forms an image on the image forming surfaces 4a and 4b. Similarly to the above, (11a and 11b in FIG. 13 or 12a and 12b in FIG. 14 are examples, and the plurality of display surfaces are not limited to these two surfaces. The edge pattern display controller 9 or 10 controls.

  First, in order to improve the multi-view three-dimensional display unit, an optical path simulation of an example in which components are arranged based on the above optical design policy was performed. When an optical system with a small F number is used as the convex lens array, an image is not formed at a fixed place at the edge of the screen, but when the F number is increased, the image forming state becomes good and a stable image is observed. (Figure 4).

  Actually, a 3840 × 2400 pixel 22-inch liquid crystal monitor as LCD, 104 square Fresnel lenses with a focal length of 90 mm and a side length of 35 mm as a convex lens array, a focal length of 600 mm and a side length of 600 mm as a Fresnel lens on the observer side. A multi-view three-dimensional display with 104 viewpoints was prepared by arranging the above lenses at a position 1000 mm from the convex lens array. As a result, it was confirmed that a clear stereoscopic image was observed in a localized position to a certain degree of stability.

  When the F number is large, the images to be displayed are densely arranged on the LCD. Therefore, a stereoscopic image cannot be displayed on a large screen unless the distance between the convex lens array and the Fresnel lens is sufficiently large. This means that an increase in the size of the apparatus is inevitable for increasing the screen size. Of course, it is conceivable to use a special lens that does not cause aberrations. However, if the individual lenses used in the convex lens array become expensive, the manufacturing cost of the entire apparatus will increase significantly. By combining a general convex lens instead of the Fresnel lens, it is possible to create an optical system that can present a large three-dimensional image with less aberration even in a short optical path.

  In addition, the hypothesis that "when there are multiple background edges with multiple depths in layers, focus adjustment is induced to the background edge presentation surface closest to the convergence angle according to the convergence angle of the parallax image to be presented." I verified it as follows. If this hypothesis is proved, congestion adjustment contradiction can be solved by simply inserting a multi-layered background edge.

  FIG. 5 shows an experimental system set up to verify this hypothesis. In this experiment, parallax is presented at a position 1 m from the observer. The parallax presentation is performed using an optical system as shown in FIG. 6 (Non-Patent Document 9), and the presented image is an image with gradation by colors with the same saturation as shown in FIG.

  First, the position of the parallax image is presented so that the angle of convergence can be guided to a depth of 40 cm from the observer, and then only one background edge is inserted from the observer at a depth of 33 cm, 40 cm, and 50 cm. The spherical power of the observer's eyes was measured with a refractometer for each case and for a case where three background edges were inserted, one for each depth. The experimental results for 7 subjects are shown in FIG. FIG. 8 shows the spherical power of the observer's eye when only one background edge is displayed and when multiple layers are displayed. As can be easily seen from FIG. 8, in the experiment in which three edge images are presented in an overlapping manner, the same result as in the case of presenting edges only at a depth of 40 cm is obtained, and the depth at which a stereoscopic image is fused by parallax is obtained. It was confirmed that the focus was guided in the vicinity.

  However, only in this experimental result, a 3D image emerged at the middle position of the three superimposed edge images. There is also an explanation that focus adjustment is induced.

  Therefore, next, while presenting a total of three background edges at the positions of 33 cm, 40 cm, and 50 cm, the parallax amount of the parallax image is changed, and the convergence is induced from the observer to a depth of 33 cm, 40 cm, and 50 cm. In this case, the spherical power of the observer's eyes was measured. The experimental results obtained for three subjects are shown in FIG. FIG. 9 shows the spherical power of the eye of the observer when the background edge of the three layers is fixed and the convergence angle is changed, and the standard value is the spherical power when observing the real index at the depth. Here, the standard value is a measurement result when an actual object (index) is placed at a corresponding depth. As shown in FIG. 9, in all cases, the result that the focus adjustment fits near the depth corresponding to the convergence angle of the parallax image was obtained.

  FIG. 15 is a diagram showing a configuration in which background edge display surfaces (13a, 13b) are installed near the display surface. The display surface 1 forms an image on the image formation surface 4, and the background edge display surfaces (13a, 13b) form an image on the image formation surfaces 4a, 4b. FIG. 16 is a diagram showing a configuration in which background edge display surfaces (14a, 14b) are provided near the imaging surface. In these figures, two background edge display surfaces are shown, but this is an example, and it is desirable to prepare more background edge display surfaces. Further, there are arbitrary patterns that can be used for the background edge display surface, and various patterns can be used. An example is shown in FIG. 17A is a vertical and horizontal grid pattern, FIG. 17B is a diagonal grid pattern, FIG. 17C is a pattern repetition, FIG. 17D is a phase-repetitive pattern, and FIG. 17E has a fine structure. (F) is a pattern in which simple figures are arranged at grid points, (g) is a pattern in which dots or lines are randomly arranged, and (h) is a composite of two patterns. . Such a pattern is drawn on a transparent glass plate, for example, and arranged as shown in FIG. 16 or FIG. These background edge display surfaces are arranged so that they can all be seen from the viewer. Background edge display surfaces having different patterns may be used for, for example, 13a and 13b, or 14a and 14b in FIG.

  Although a convex lens array has been described above, this can be extended to a condensing system array generally having a focus. Further, although a Fresnel lens is used as a large-diameter convex lens, there is no problem even if this is extended to a large-diameter condensing system generally having a focus. In addition, it is obvious that the above-described condensing system array can be applied to the present invention if it functions as a condensing system array as a result of combining a plurality of lens arrays. Similarly, it is obvious that the large aperture condensing system may be configured by combining a lens or a mirror. FIG. 10 shows an example in which aberration is suppressed by using two convex lenses for each condensing system array and two convex lenses for the large-diameter condensing system.

  In the above description, it is easy to understand that the condensing system array and the large-diameter condensing system have a fixed focal length. However, there is no problem even if these light collecting systems are variable focus light collecting systems. A more suitable display can be performed by using a variable focus condensing system and by slightly changing the focus characteristics for each scene.

  Further, in the above description, only one person is shown as an observer. However, even if the position of the observer is changed, the above description is valid, so for many observers, the multi-viewpoint of the present invention can be achieved. Obviously, the stereoscopic image of the stereoscopic display device can be observed simultaneously.

It is a conceptual diagram which shows multiview FLOATS. It is a figure for demonstrating why a fly-eye lens is needed in multiview FLOATS. It is a figure which shows arrangement | positioning of an edge image. It is a figure which shows that when the F number is increased, the image formation state becomes good and a stable image is observed. It is a figure which shows the experimental system assembled in order to verify a hypothesis. It is a figure which shows the optical system used for parallax presentation. It is a figure which shows the presentation image used again for parallax presentation. It is a figure which shows the spherical power of an observer's eye at the time of displaying the case where only one background edge is displayed and multilayer display. It is a figure which shows the spherical power of an observer's eye at the time of fixing the background edge of 3 layers and changing a convergence angle. It is a figure which shows the example which suppressed aberration using two convex lenses for each condensing system array, and using two convex lenses also for a large aperture condensing system. It is a block diagram which shows the structure of this invention. It is a block diagram which shows the structure of this invention which multiplies an image display surface and forms a multilayer image plane. It is a block diagram which shows the structure of this invention which controls the edge pattern display surface near a display surface with an edge pattern display controller. It is a block diagram which shows the structure of this invention which controls the edge pattern display surface near an imaging surface with an edge pattern display controller. It is a block diagram which shows the structure which installed the background edge display surface near the display surface. It is a block diagram which shows the structure which installed the background edge display surface near the image plane. It is a figure which shows the example of a pattern of a background edge.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Display surface 2 Condensing system array 3 Large aperture condensing system 4 Imaging surface 4a, 4b Imaging surface 5 Observer 6 Display controller 7a, 7b Display surface 9, 10 Edge pattern display controller 11a, 11b Edge pattern display Surface 12a, 12b Edge pattern display surface 13a, 13b Background edge display surface 14a, 14b Background edge display surface

Claims (7)

A display surface for displaying pixels;
A plurality of condensing systems that enter the light from the display surface and emit parallel light, and are formed adjacent to each other along the display surface; and
A large-diameter condensing system that injects parallel rays from the condensing system array and emits the light toward each of both eyes of the observer;
On a line connecting the display surface and the observer, a plurality of display surfaces provided at positions before or after the display surface at a position farther than the light collection system array as viewed from the observer , or the above On the line connecting the display surface and the observer, a plurality of display surfaces provided between the large-diameter condensing system and the observer and provided at a position before or after the imaging surface of the large-diameter condensing system , With
The parallel light beam emitted from the condensing system does not form an image between the condensing system array and the large-diameter condensing system, but forms an image between the large-diameter condensing system and the observer. Parallel rays at
The multi-view stereoscopic display device characterized in that the large-diameter condensing system forms an image on an image plane between the large-diameter condensing system and the observer.   The multi-viewpoint three-dimensional display device according to claim 1, wherein the condensing system array is a convex lens array.   The multi-view stereoscopic display device according to claim 1, wherein the light collection system array is a light collection system array in which a plurality of lenses are combined.   The multi-view stereoscopic display device according to claim 1, wherein the large-diameter condensing system is a convex lens.   The multi-view stereoscopic display device according to claim 1, wherein the large-diameter condensing system is a condensing system in which a plurality of lenses or mirrors are combined. Displaying three-dimensional image information using the display surface and the plurality of display surfaces,
An image corresponding to the depth position of the image displayed on the display surface is displayed on the plurality of display surfaces,
The multi-view stereoscopic display device according to any one of claims 1 to 5, further comprising a configuration in which an image on the display surface and a plurality of further provided display surfaces are visible to the observer. On the line connecting the display surface and the viewer, at a position farther than the condensing system array as viewed from the viewer and before or after the display surface , or the display surface and the viewer The edge pattern of the image displayed on the display surface is displayed at a position between the large aperture condensing system and the observer and before or after the image plane of the large aperture condensing system. Providing a plurality of edge pattern display surfaces to display a predetermined fine edge pattern on the edge pattern display surface corresponding to the depth position of the image,
6. The multi-viewpoint three-dimensional display device according to claim 1, further comprising a configuration in which all of the image on the display surface and the edge pattern on the edge pattern display surface are visible to the observer.

Images