US20110188116A1

US20110188116A1 - Device for generation of three-demensional images

Info

Publication number: US20110188116A1
Application number: US12/931,147
Authority: US
Inventors: Nikolay Ledentsov Ledentsov
Original assignee: Individual
Current assignee: Individual
Priority date: 2010-02-02
Filing date: 2011-01-26
Publication date: 2011-08-04
Also published as: US10205935B2; US20160127722A1; US9936193B2; US20160261858A1; US20170332071A1

Abstract

A device for generation of three-dimensional images in the observer's eye using at least one two-dimensional display and an optical system with an adjustable focal length. The display generates planar images corresponding to the different depths of the designed three dimensional (3D) image and the images are then directed to at least one mirror or lens with the adjustable focal length synchronized to the particular image. The optical system generates virtual images at different virtual planes in such a way that the eye receives a complete 3D virtual image.

Description

This application claims the benefit of U.S. Provisional Application No. 61/337,263, filed Feb. 2, 2010

BACKGROUND OF THE INVENTION

1. Field of the Invention
The present invention relates to a device generating virtual three dimensional (3D) images, preferably moving images, formed from planar two-dimensional (2D) images. More specifically the present invention relates to a device for generation of virtual 3D moving images such as 3D video or motion pictures.
2. Description of Related Art
There is a need in generation of three-dimensional moving images. Possible applications include 3D: cinema; TV displays, out-of-door advertisement boards; computer monitors for training and education; mobile phone projectors, head-up-displays, interactive displays, “touch screen” displays and other. There are different approaches for generation of 3D images.
Some of the approaches are based on the delivery of two different images generated on a surface of a special screen separately to the observer's left and right eyes thus generating a stereoscopic image. This goal may be achieved by glasses, for example, using blinking glasses synchronized with the display sequentially demonstrating the “left” and the “right” images on the screen. In this case the left and the right eye get the corresponding images separately. In another approach the glasses separate the images for the left and the right eye by using polarized glasses (different linear or circular polarization for the left and the right eyes) and the two polarized images created on the polarization-maintaining screen. It is also possible to generate images on the conventional screen using two slightly different wavelengths in the red, green and blue spectral ranges and then apply glasses with the corresponding spectrally-sensitive filters to let each eye receive its own full color image based on the related selected wavelengths.
In another approach a 3D stereoscopic image is realized by using lens gratings attached to the screen, and an image composed of the alternating stripes of the “left eye” and “right eye” images with the same special periodicity as the periodicity of the lens grating. The lens grating properly attached to the screen deflects the images in a narrow angle space to match the left and the right eye separately. However, the viewing angle range is very narrow in this approach making it so far less suitable for major applications.
In another approach the real 3D object is created by the rotation or motion of a screen or a frame and the synchronized generation of images corresponding to different 3D image planes occurs. In this approach the signal generation is synchronized with the rotation speed. In this way a 3D image is generated in the eye without any need in special glasses providing an immense advantage over the glass-related approaches. Special data processing algorithms are developed to generate a 3D image in real time, for example by processing the data derived from stereoscopic or holographic 3D images. For example, left eye and right eye images from conventional glass-assisted 3D images can be processed, coded to a new approach and used to generate 3D images in the moving screen approach.
While suited for their intended purpose, a disadvantage of devices of this type is that the display must move at a high rotation or one-directional motion speed. Furthermore the most common rotary 3D displays require complex and highly precise mechanics and the related precisely synchronized display imaging schemes.
On the other hand an approach for generating of virtual 3D images from real 3D objects is known and used in entertainment and for educational purposes for many centuries already and can serve as a prior art prototype for the present invention.

SUMMARY OF THE INVENTION

An object of the present invention aims elimination of the above-mentioned disadvantages by providing a virtual image device generating 3D images in the eyes of the observer using conventional displays, based, for example, on conventional DLP or LCOS microdisplay technologies, or the screens with projected images of the related microdisplays. Furthermore, liquid crystal (LCD), plasma (PD) or organic LED (OLED) displays can be used for 3D image generation.
The disadvantages of the previous 3D motion display approaches can be eliminated by the features of the virtual image device of the present invention which comprise at least one lens or mirror, or their combination, capable to generate virtual images at different virtual planes. When the images are generated at different virtual image planes correspond to the different depths of the designed virtual 3D image, and the time duration for the image generation is shorter than the eye time resolution, a complete 3D virtual image can be formed in the observer's eye. For example, the images at different image depth can be generated at the 2D display, while the lens system will generate virtual images at different virtual image planes directly in the eye of the observer. No need in the display motion is required (even can be also applied) and any conventional display approach can be used. A very high quality of the moving 3D image can be realized. Plasma, OLED, DLP, LCD or CRT displays may generate initial images. A synchronized motion of the lens and/or mirror can be needed and a motion of the 2D displays can be additionally applied. As the lenses with an adjustable focus may have a rather small weight and volume, the motion is simplified. In certain cases the lenses or mirrors may not move, while electro-optic or acousto-optic or piezo-optic effects may be applied to change the focal lengths. Furthermore, microdisplay-based adjustable lenses may be applied. For example, standard DLP micro-electro-mechanic system (MEMS) technology may be used to cause tunable focal length of the lense or a tunable shape of the reflecting mirror. Liquid or other deformable lenses and mirrors can be also applied. In the case when the focal length is sensitive to the wavelength, for example, when diffraction gratings are introduced or wavelength-dispersive medium is used for the lenses or mirrors no adjustments of lenses or mirrors are needed and the 3D image may be generated by wavelength tuning of the light used for the generation of the images.

BRIEF DESCRIPTION OF THE DRAWINGS

The key objects, advantages, and features of the invention will become apparent from the foregoing detailed description taken in connection with the accompanying drawings, in which:

FIG. 1

A virtual image of the object originating in the eyes in prior art device.

FIG. 2

Prior art device. An internal arrangement of the elements in a typical prior-art device is shown. A real object illuminated by a lamp and the light coming from the object is reflected by a curved mirror through a window to form a virtual image in the observer's eyes. The observer is looking at the window form outside, as depicted in FIG. 1.

FIG. 3

A device similar to shown in FIG. 2 where the real object is replaced by a display. The display moves in between of the two or more positions. The image on the display is changed when the display is moved. In this case a virtual three-dimensional image can be generated in the eye from a set of two-dimensional images if the motion is faster than the eye time-resolution.

FIG. 4

(a) is a schematic view of a virtual image device according to the prior art invention; a lens is used to generate the virtual image.

(b) is a schematic view of a virtual image device according to the prior art invention; a mirror is used to generate the virtual image.

(c) is a schematic view of a virtual image device according to the prior art invention; a mirror is used to generate the virtual image. The virtual image is enlarged as compared to the real object.

FIG. 5 is a schematic view of a virtual image device according to the present invention; a focus-adjustable mirror is used to generate the virtual image at different depths of the resulting three-dimensional (3D) virtual image. The virtual two-dimensional (2D) images are positioned at different virtual planes when the focus of the adjustable lens is affected (a, b).

FIG. 6 is a schematic view of a virtual image device according to the present invention; a focus-adjustable mirror is used to generate the virtual image at different depths of the resulting three-dimensional (3D) virtual image. The virtual two-dimensional (2D) images are positioned at different virtual planes when the focus of the lens is affected due to the different wavelengths of the light used to generate the images (a, b).

FIG. 7 is a schematic representation of the interactive system composed of: a projection display generating virtual image in the observer's eye; a real object; a camera suitable for taking three-dimensional pictures and an image-processing system, which aligns the geometrical characteristics of a real object with the geometrical characteristics of the virtual image allowing to obtain necessary interactivity by aligning real and virtual images according to the selected algorithm.

DETAILED DESCRIPTION OF THE INVENTION

The virtual 3D image production device is shown in FIGS. 1-7 for purposes of illustration. In FIG. 1 we show an eye appearance of a three-dimensional object in a prior art device generating virtual images. A high-quality virtual image hardly distinguishable from the original real object may be generated.
FIG. 2 shows a schematic cross-section of the prior art device. A virtual image of the object is originating in the observer's eye as illustrated in FIG. 1 due to a system composed of: a real object; a lamp; and a curved mirror. The light from the real object illuminated by a lamp is reflected by a curved mirror through a window and a virtual image is formed for the observer's eyes. The observer is looking at the window form outside, as depicted in FIG. 1.
A device similar to shown in FIG. 2 where the real object is replaced by a two-dimensional display is shown in FIG. 3. The display moves in between of two or more positions. The image on the display is changed when the display is moved. In this case a virtual three-dimensional image can be generated in the eye from a set of two-dimensional images if the motion is faster than the eye averaging time and the image sequence is synchronized with the motion speed in an appropriate way.
In FIG. 4 we show schematically a sketch of the optical system of prior art device which generally includes at least one lens (1) or a mirror (2) positioned with respect to the optical axis (3) and having a finite or infinite focus (F) length (4). An object (O) is positioned (5) with respect to the optical axis and lens(es) or mirror(s). The optical system results in a virtual image I (6). Optical system may be arranged in such a way that the virtual image may be reduced (FIG. 4 a, b) or enlarged (FIG. 4 c) with respect to the object or a real image.
The properties of the lens or the mirror used can be either movable, or deformable or modified by acousto-optic, electrooptic, piezo-optical, thermal or other suitable effects. Many types of tunable-focus lenses and mirrors are known and their present and future modifications can be used.
In FIG. 5 one illustrates the case when the mirror is deformable and this affects the actual focus length. Then the virtual image plane will move in space. Consequently, when the images corresponding to the different 3D image depths are generated by the display these can be properly positioned by the synchronized movable mirror system. In addition to the adjustment of the lenses or mirrors the screen motion can be also applied. The motion of the lenses and mirrors and the display may occur in all three dimensions when necessary.
FIG. 6 illustrates another embodiment of the present invention. The length chosen in this case is neither movable nor deformable. However, the focal length of the lens is wavelength-sensitive, for example, due to the related refractive index dependence of the material used in the lens or due to diffraction granting introduced or any other approach or their combination. Then the images can be separated in depth by projecting different depth images in slightly different colors for the same basic color range. Then separate virtual 3D images in blue, red and green spectral ranges may be generated. These images may be also fused into the final full-color virtual 3D image using any of the image fusion techniques, for example, using additional diffraction gratings as angle-sensitive mirrors.
Furthermore two separated 2D images for the left and the right eye, originating from two different displays, may be fused into a one virtual image using a similar approach adding a possibility of stereoscopic imaging without using lens arrays attached to the display. This approach may provide additional flexibility and complexity to the device.
In all of the embodiments, the formation of a virtual 3D image from different 2D image planes should preferably occur within the finite time response resolution of the human eye.
In FIG. 7 we show a schematic representation of an interactive system composed of a projection display generating virtual image in the observer's eye, a real object, a camera suitable for taking three-dimensional pictures and an image-processing system, which aligns the geometrical characteristics of the real object with the geometrical characteristics of the virtual image and allows to obtain the necessary interactivity by image recognition data processing techniques according to the selected algorithm.
For example, a virtual keyboard can be projected at a certain virtual distance with respect to the operator's eyes, for example in front of the face of the operator or on top of the desk in front of the operator. The hands of the operator are monitored by a three-dimensional camera. In one embodiment only the virtual image of the keyboard is projected. In another embodiment also a three-dimensional virtual image of the operator's hands is projected. The latter version is useful when the operator observes simultaneously a real or another virtual image. For example, a driver or a pilot may need this approach to change the parameters of the motion or communicate certain information without deflection from a window or a screen or another virtual image generated in his eyes. The projector and the camera are connected to the image processing system which aligns virtual keyboard and a real or a virtual hand of the operator and the necessary functionality is achieved. For example, typing of symbols or letters, or drawing figures onto a virtual screen can be realized.
The system may be useful for automotive, airspace, gaming and other industries and can be broadly used in mobile phones and other gadgets.
While the present invention has been described in connection with the preferred embodiments, it is to be understood that the invention is not to be limited to the disclosed embodiments, but to the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit of the invention, which are set forth in the appended claims, and which scope is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures.

Claims

1. The device for generation of three-dimensional images using at least one display and an optical system, wherein the display generates two-dimensional (2D) images corresponding to the different depths of the final 3D image and the images are then directed to at least one mirror or lens of the optical system with an adjustable focal length, generating virtual images at different virtual planes in such a way, that the eye receives a complete three dimensional virtual image.

2. The device of claim 1 where the adjustment of the focal length of the lens or mirror is achieved due to one of the following approaches or their combination:

mechanical motion of the lens

electrooptic effect-induced change of the refractive index and the focal length

thermal effect-induced change of the refractive index and the focal length

applying acousto-optic effect-induced change of the refractive index and the focal length

electrostatic, or electric field-induced change of the shape of the lens or mirror

applying reconfigurable lens or mirror based on the micro-electro-mechanic system (MEMS)

In all the cases using conventional or Fresnel lenses or mirrors can be applied.

3. The device of claim 1 where the adjustment of the focal length of the lens or mirror for the realization of virtual images at different virtual focal planes is achieved by selecting different wavelength according to one of the following approaches or their combination:

wavelength-sensitive focal length of the lens or mirror

applying multiple-layer mirrors with different foci with narrow reflectivity bands, which reflect only particular wavelengths.

4. The device of claim 1 where several virtual 3D images are generated by several displays.

5. The device of claim 4 where several virtual 3D images are merged into a single virtual image by applying an additional optical system.

6. The device of claim 4 where the two or more different planar display images separated in space in the same plane or in different planes are combined into a single virtual image.

7. The device of claim 6 where 2D images corresponding to different image planes are generated by scanning of the light beams at different parts of the screen and have different focal distances with respect to the lens or mirror while the optical system merges the spatially-separated images into a single virtual 3D image or multiple virtual 3D images.

8. The device of claim 1 wherein said device is one of the group: a film projector, a slide projector, a TV projector, an outdoor advertisement projector, a head up display, a mobile display.

9. The device of claims 1-8, where two dimensional images are generated by a device chosen from a group: plasma display, liquid crystal display, LED display, organic LED display, CRT, or a lamp; LED or a laser projection display.

10. An interactive system composed of a projection display generating virtual image in the observer's eye of claims 1-9, a real object, a camera suitable for taking three-dimensional pictures and an image-processing system, which aligns the real object with the geometrical characteristics of the virtual image and allows to obtain the necessary interactivity by aligning the real and the virtual images according to the selected algorithm