CN205787364U - Near-eye display device - Google Patents

Abstract

The utility model provides a near-eye display device, which comprises an illumination module and a light modulation element: the illumination module is used for outputting images; the light modulation element is used for integrated imaging of the image to display a three-dimensional virtual image. The near-eye display device may include a near-eye display device for virtual reality. The near-eye display device uses the principle of integrated imaging display to present a natural three-dimensional object to the human eye, which solves the problem of visual fatigue caused by long-term viewing of three-dimensional images. It is especially important for application scenarios where the device needs to be worn for a long time.

Description

near-eye display device

technical field

The utility model relates to the technical field of terminal equipment, in particular, the utility model relates to a near-eye display device.

Background technique

AR (Augmented Reality, Augmented Reality) display device can superimpose virtual objects in the real scene, which applies virtual information to the real world and is perceived by human senses, so as to achieve a sensory experience beyond reality, that is, the real environment and virtual The objects are superimposed on the same screen or space in real time and exist at the same time.

An existing display technology provides a display device capable of improving light efficiency and viewing angle. As shown in FIG. 1 , the device includes: an illumination module, a first polarization beam splitter, a polarization rotator, and an end reflector. Wherein, the lighting module 206 includes: a display panel 250 , a scattering surface 255 , a second polarizing beam splitter, a light source, and a converging lens 240 . The light from the light source is directed to the second polarized beam splitter through the converging lens, the light in the S polarized state is reflected by the second polarized beam splitter to the display panel 250, and becomes the light in the P polarized state after being reflected by the display panel 250, and the light in the P polarized state Through the second polarization beam splitter, the first polarization beam splitter and the polarization rotator, it is transmitted to the end reflector, and then the end reflector reflects the light back. After passing through the polarization rotator, the light becomes S-polarized light, and the S-polarized state The light is reflected to the human eye through the first polarizing beam splitter, forming a 3D image. When the light from the light source passes through the converging lens to the second polarization beam splitter, the light in the P polarization state is transmitted to the scattering surface 255 by the second polarization beam splitter, and the scattering surface scatters this part of the light. In this solution, the light efficiency can be improved through the polarizing beam splitter, and the viewing angle can be improved through the concave reflector. However, the display device cannot display natural three-dimensional objects, which causes visual fatigue of human eyes when viewing stereoscopic images; therefore, wearing such a display device for a long time is not conducive to human eye health, and the user's wearing experience is poor.

To sum up, existing related display devices have the problem of easily causing visual fatigue of human eyes when displaying three-dimensional virtual objects.

Utility model content

The purpose of this utility model is to at least solve one of the above-mentioned technical defects, especially the problem of visual fatigue of human eyes easily caused when displaying three-dimensional virtual objects.

The utility model provides a display device for augmented reality, including a lighting module and a light modulation element:

The lighting module is used to output images;

The light modulation element is used for integrated imaging of the image to display a three-dimensional virtual image.

Wherein, the light modulation element includes a microlens array or a microhole array; the microlens array or the microhole array may be a curved microlens array or a curved microhole array. The light modulating element may include a dynamic microlens array or a microhole array composed of liquid crystal elements.

Preferably, the lighting module includes at least two display panels and light splitting devices:

Specifically, at least two display panels are located on both sides of the light splitting device and form a predetermined angle with the light splitting device;

Wherein, the display panel is used for displaying images, and the light splitting device is used for transmitting the images displayed by the display panel.

Preferably, at least two display panels form an angle of 45 degrees with the light splitting device.

Optionally, when the display panel is a non-self-illuminating panel, the lighting module further includes a light source and a converging mirror:

The converging mirror is located between the light source and the display panel, and the light emitted by the light source illuminates the display panel through the converging mirror.

Preferably, the beam splitting device is a polarization beam splitting device: the polarization beam splitting device reflects the light of the first polarization direction component collimated by the converging mirror from the light source, and transmits the light of the second polarization direction component orthogonal to it, the transmission and The reflected polarization direction of light is used for illumination of non-self-illuminating panels.

Optionally, when the display panel is a self-luminous panel, the light splitting device transmits and reflects light from the self-luminous panel.

Optionally, when the display panel is a monochromatic self-luminous panel, the light splitting device is a band-pass color light-splitting device: the band-pass color light-splitting device reflects light of the same color as the monochromatic self-luminous panel, and transmits light of other colors .

Preferably, at least two display panels switch to display images at a frequency greater than a predetermined refresh rate, and the images displayed by each display panel are staggered by a predetermined proportion of pixels in the horizontal and vertical directions.

More preferably, when the number of display panels is n, n display panels switch and display images at a frequency greater than n×30 Hz, and the images displayed by each display panel are staggered by 1/n pixels horizontally and vertically .

Preferably, the light modulation element is located in the lighting module at a position at a first predetermined distance from each display panel.

Preferably, the display device further includes a light conduction unit, the light conduction unit includes at least one lens; the light conduction unit guides the image displayed on the display panel to a position at a predetermined distance from the light modulation element for use in the light modulation element Perform integrated imaging.

Optionally, when the light conduction unit includes two lenses, at least one display panel is located at one focal length of one lens, and the distance between the two lenses is twice the focal length; A second predetermined distance from the location.

Optionally, when the light conduction unit includes a lens, at least one display panel is located at twice the focal length of the lens; the light modulation element is located at a third predetermined distance from the imaging position after the image is transmitted through the light conduction unit.

Preferably, a reflective element is also included: the reflective element is located in the direction of the light path of the lighting module, and guides the three-dimensional virtual image to human eyes.

Wherein, the reflective element includes a mirror or a beam splitter.

Preferably, when the near-eye display device is a near-eye display device for augmented reality, the near-eye display device further includes a correction module, and the reflective element is a beam splitter;

The beam splitter divides the light of the three-dimensional virtual image and the real image of the outside world into two paths, which are directed to the human eye and the correction module respectively;

The correction module is used to correct the three-dimensional virtual image based on the three-dimensional virtual image transmitted by the beam splitter and the real image of the outside world, and display the corrected three-dimensional virtual image through the lighting module.

Preferably, the calibration module includes:

An image capture unit, configured to acquire a three-dimensional virtual image from the spectroscope and an external real image;

The correction unit is used to analyze the three-dimensional virtual image and the real external image, and correct the three-dimensional virtual image according to the analysis result;

The image rendering unit is used for rendering the 3D virtual image obtained after correction.

More preferably, the correction module also includes:

The light source control unit is used to adjust the brightness of the light emitted by the light source according to the three-dimensional virtual image obtained after correction.

The utility model provides a near-eye display device. For example, the near-eye display device can be a near-eye display device for virtual reality. The near-eye display device utilizes the principle of integrated imaging display to make a natural three-dimensional object presented to the human eye. It solves the problem of visual fatigue caused by long-term viewing of three-dimensional images, which is especially important for application scenarios that require long-term wearing of this device; further, if the near-eye display device is a near-eye display device for augmented reality, the near-eye display device can be based on real-time The matching between the obtained external real image and the three-dimensional virtual image is corrected and adjusted in real time and the three-dimensional virtual image is rendered, which improves the function of the near-eye display device. Furthermore, the near-eye display device can use multiple display screens to improve display quality; meanwhile, the polarization beam splitter can be used to improve light efficiency. At the same time, the above-mentioned solution proposed by the utility model has little modification to the existing system, does not affect the compatibility of the system, and is simple and efficient to implement.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description, or may be learned by practice of the invention.

Description of drawings

The above-mentioned and/or additional aspects and advantages of the utility model will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

Fig. 1 is a device structure schematic diagram of a kind of existing display technology in the utility model;

FIG. 2 shows a schematic diagram of a display device for virtual reality in the first embodiment of the present invention;

Fig. 3 shows a device schematic diagram of a display device for augmented reality in the present invention;

FIG. 4 is a schematic diagram of another display device for augmented reality in the present invention;

Fig. 5a shows a schematic structural view of the first embodiment of the lighting module in the present invention;

Fig. 5b shows a schematic structural view of the second embodiment of the lighting module in the present invention;

Fig. 5c shows a schematic structural view of the third embodiment of the lighting module in the present invention;

Fig. 5d shows a schematic structural view of the fourth embodiment of the lighting module in the present invention;

Figure 6a shows a schematic structural view of the first embodiment of the light transmission unit in the present invention;

Fig. 6b shows a schematic structural view of the second embodiment of the light transmission unit in the present invention;

Fig. 7 shows the structural representation of plane lens array in the utility model;

Fig. 8 shows the structural representation of the curved surface lens array in the utility model;

Fig. 9 shows the structural representation of the microhole array in the utility model;

Fig. 10 shows a schematic diagram when human eyes watch a two-dimensional image;

Fig. 11 shows a schematic diagram when human eyes watch a common three-dimensional image;

Fig. 12 shows a schematic diagram of human eyes viewing the integrated imaging display in the present invention;

Fig. 13 shows a schematic diagram of binocular near-eye light field display in the utility model;

Fig. 14 shows a display device for augmented reality in the second embodiment of the present invention;

FIG. 15 shows a display device for augmented reality according to a third embodiment of the present invention;

Fig. 16 shows a display device for virtual reality in the fourth embodiment of the present invention;

Fig. 17 shows a schematic flowchart of a three-dimensional augmented reality embodiment of the present invention;

FIG. 18 is a schematic flow diagram of a three-dimensional augmented reality engine in a preferred embodiment of the present invention;

Fig. 19 is a schematic flow chart of three-dimensional light field rendering in a preferred embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are only used to explain the present invention, and cannot be construed as limiting the present invention.

Those skilled in the art will understand that unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the word "comprising" used in the description of the present utility model refers to the presence of the stated features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features , integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The expression "and/or" used herein includes all or any elements and all combinations of one or more associated listed items.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as commonly understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have meanings consistent with their meaning in the context of the prior art, and unless specifically defined as herein, are not intended to be idealized or overly Formal meaning to explain. It should be noted that the near-eye display device of the present invention includes a virtual reality display device or an augmented reality display device.

In one embodiment of the present invention, the display device includes an illumination module and a light modulation element; wherein, the illumination module outputs an image; the light modulation element performs integrated imaging on the image to display a three-dimensional virtual image. In this embodiment, a natural three-dimensional object is presented to human eyes by using the principle of integrated imaging display, which solves the problem of visual fatigue caused by long-term viewing of three-dimensional stereoscopic images.

Preferably, the display device further includes a reflective element: the reflective element is located in the direction of the light path of the lighting module, and guides the three-dimensional virtual image displayed by the light modulation element to human eyes. Wherein, the reflective element includes a mirror or a beam splitter.

Fig. 2 shows a display device for virtual reality according to the first embodiment of the present invention, and the display device includes an illumination module, a light modulation element and a mirror. In this embodiment, the light modulation element processes the image displayed on the display panel into a virtual object light field of a three-dimensional virtual image, and projects the three-dimensional virtual image to human eyes through a beam splitter. Through this embodiment, human eyes can observe virtual three-dimensional objects, thereby realizing the display of virtual reality.

Preferably, the light modulation element is located in the lighting module at a position at a first predetermined distance from each display panel.

It should be noted that the structure of the display device for virtual reality shown in FIG. 2 can be used for a display device for augmented reality. In this case, the spectroscopic element may be a spectroscopic mirror.

Preferably, the near-eye display device further includes a light conduction unit, and the light conduction unit includes at least one lens; the light conduction unit conducts the image displayed on the display panel to a position at a predetermined distance from the light modulation element, so that the light modulation element performs integrated imaging . Preferably, as shown in FIG. 3, when the near-eye display device is a display device for augmented reality, the display device further includes a correction module, and the reflective element is a beam splitter;

The beam splitter divides the light of the three-dimensional virtual image and the real image of the outside world into two paths, which are directed to the human eye and the correction module respectively;

The correction module corrects the three-dimensional virtual image based on the three-dimensional virtual image transmitted by the beam splitter and the real external image, and displays the three-dimensional virtual image obtained after correction through the lighting module.

Preferably, as shown in FIG. 4 , the correction module includes an image capture unit, a correction unit and an image rendering unit; the image capture unit may be a camera.

The image capture unit acquires the 3D virtual image from the spectroscope and the external real image; the correction unit analyzes the 3D virtual image and the external real image, and corrects the 3D virtual image according to the analysis results; the image rendering unit corrects the corrected 3D virtual image to render.

After changing the beam splitter in the display device for augmented reality in Figure 3 to a mirror, or after changing the beam splitter in Figure 4 to a mirror and removing the image capture unit, the display device for augmented reality can be changed A display device for virtual reality.

More preferably, the correction module further includes a light source control unit; the light source control unit adjusts the brightness of light emitted by the light source according to the three-dimensional virtual image obtained after correction. For example, when the display panel is a self-illuminating display panel, the image displayed on the display panel is transmitted to the light conduction unit through the light splitting device, and the light conduction unit images the image displayed on the display panel at a predetermined distance from the light modulation element, thereby forming an integrated imaging 3D virtual image displayed. The displayed virtual object light field enters the human eye and the image capture unit respectively through the beam splitter. The human eye and the image capture unit can simultaneously acquire real images of the outside world. The image capture unit can receive the same image content as observed by the human eye, that is, the augmented reality display image superimposed between the 3D virtual image and the real external image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit Perform real-time re-rendering to achieve the purpose of adjusting the 3D virtual image in real time.

For another example, when the display panel is a non-self-illuminating display panel, the illumination light emitted by the light source illuminates the display panel through the light splitting device, and the light conduction unit images the image displayed on the display panel at a position at a second predetermined distance from the light modulation element, thereby forming an integrated Imaging display. The light modulation element processes the image displayed on the display panel into a three-dimensional virtual image displayed by integrated imaging, and the virtual object light field of the three-dimensional virtual image enters the human eye and the image capture unit respectively through the beam splitter. The human eye and the image capture unit can simultaneously acquire real images of the outside world. The image capture unit can receive the same image content as observed by the human eye, that is, the augmented reality display image superimposed between the 3D virtual image and the real external image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit Perform real-time re-rendering to achieve the purpose of adjusting the 3D virtual image in real time.

The implementation of each component of the first embodiment will be described respectively below.

1. Lighting module

The lighting module includes at least two display panels and a light splitting device; at least two display panels are located on both sides of the light splitting device and form a predetermined angle with the light splitting device; preferably, at least two display panels form a 45-degree angle with the light splitting device.

Wherein, the display panel displays images, and the light splitting device conducts the images displayed on the display panel.

Optionally, when the display panel is a non-self-illuminating panel, the lighting module further includes a light source and a converging mirror: the converging mirror is located between the light source and the display panel, and the light emitted by the light source illuminates the display panel through the converging mirror.

Optionally, when the display panel is a self-luminous panel, such as an OLED (Organic Light-Emitting Diode, organic light-emitting diode) display panel, the lighting module does not need an additional light source, and the light splitting device transmits and reflects light from the self-luminous panel.

Optionally, when the display panel is a monochromatic self-luminous panel, the light splitting device is a band-pass color light-splitting device: the band-pass color light-splitting device reflects light of the same color as the monochromatic self-luminous panel, and transmits light of other colors .

Preferably, at least two display panels switch to display images at a frequency greater than a predetermined refresh rate, and the images displayed by each display panel are staggered by a predetermined proportion of pixels in the horizontal and vertical directions. For example, the resolution can be doubled by using two display panels to switch display images with a refresh rate greater than a predetermined one, and the images of each display panel are staggered by 1/2 pixel horizontally and vertically.

More preferably, when the number of display panels is n, n display panels switch and display images at a frequency greater than n×30 Hz, and the images displayed by each display panel are staggered by 1/n pixels horizontally and vertically . For example, n display panels switch to display images at a frequency of n×60 Hz, so that human eyes do not perceive flicker.

The lighting module in this embodiment includes any implementation scheme:

Solution 1: The lighting module uses multiple display panels, and the multiple display panels are placed around the light splitting device and form a predetermined angle with the light splitting device, for example, at an angle of 45 degrees. As shown in FIG. 5a, there may be two display panels, and the light splitting device may be a polarization beam splitter. The polarization beam splitting device reflects the light of the first polarization direction component collimated by the converging mirror from the light source, and transmits the light of the second polarization direction component orthogonal to it, and the transmitted and reflected polarization direction light is used for non-self Illumination with luminous panels.

After the illumination light from the light source passes through the converging mirror, it is collimated into parallel light, and is directed to the polarizing beam splitter at an angle of 45 degrees with the polarizing beam splitter. The polarizing beam splitter reflects the light of the first polarization state in the illumination light to the display panel 1 to provide illumination light for the display panel 1 , and transmits the light of the second polarization state to the display panel 2 to provide illumination light for the display panel 2 . The light in the first polarization state is reflected by the display panel 1 and becomes light in the second polarization state, which carries display image information, for example, the image displayed on the display panel 1 is transmitted through the polarization beam splitter. The light of the second polarization state is reflected by the display panel 2 and becomes light of the first polarization state, carrying display image information, for example, the image displayed by the display panel 2 is reflected by the polarization beam splitter.

In this embodiment, by using two display panels and polarizing beam splitters, the illumination light from the light source will not be reflected back to the position of the light source or scattered, and all the light will be used to illuminate the display panel and be provided to the subsequent optical path, thereby effectively The utilization efficiency of the illumination light is improved, and the light loss and energy consumption are reduced compared with the method of using the scattering film to scatter part of the illumination light in the prior art, and the battery life of the device can be effectively improved.

The method of time-division multiplexing of two display panels is specifically as follows. Two display panels are used to display image content, and the two display panels switch to display images at a rate greater than a predetermined refresh rate. The predetermined refresh rate is much higher than the refresh rate that can be distinguished by human eyes. For example, The predetermined refresh frequency is greater than 2×30Hz=60Hz, and the displayed images are staggered by half the pixel distance of the display panel in the horizontal and vertical directions, so that the resolution of the displayed image will be doubled when the human eye synthesizes it. The principle of time-division multiplexing of n display panels is similar to the method of time-division multiplexing of two display panels. The predetermined refresh frequency of each panel is greater than n×30Hz, and the images of each display panel are staggered by 1/n horizontally and vertically. pixels, thus increasing the image resolution by n times.

In this solution, on the one hand, the time-division multiplexing method of multiple display panels is used to improve the resolution of the displayed image, which can provide good display quality;

Solution 2: The lighting module uses multiple display panels, and the multiple display panels are placed around the light splitting device and form an angle of 45 degrees with the light splitting device. The beam splitting device does not use a polarizing beam splitter, but only uses a common beam splitter, as shown in Figure 5b, where there can be two display panels, and the beam splitting device can be a common beam splitter.

After the illumination light from the light source passes through the converging mirror, it is collimated to become parallel light, and then shoots to the beam splitter at an angle of 45 degrees with the beam splitter. A part of the light is reflected to the display panel 1 to provide illumination light for the display panel 1 , and another part of the light is transmitted to the display panel 2 to provide illumination light for the display panel 2 . The illumination light is reflected by the display panel 1, carries the display image information of the image, is transmitted through the beam splitter, and enters the light transmission unit; the other part of the illumination light is reflected by the display panel 2, carries the display image information of the image, is reflected by the beam splitter, and enters the light transmission unit. conduction unit. As a result, the resolution of the displayed image will be improved as in the above scheme 1 by adopting the time division multiplexing method, but the utilization efficiency of the illumination light will be lower than the above scheme 1.

Solution 3: The lighting module uses multiple OLED panels, and the multiple OLED panels are placed around the light splitting device and form an angle of 45 degrees with the light splitting device. As shown in Figure 5c, two OLED panels are used, and the light splitting element is a common beam splitter.

The light carrying images and displaying image information emitted by the two OLED panels is respectively reflected and transmitted by the beam splitter and enters the light transmission unit. A scattering surface is used to scatter the unutilized light emitted by the OLED panel after being reflected by the spectroscope and transmitted to the scattering surface, so that it will not enter the transmission light path, thereby avoiding the influence of background noise on the displayed image.

In this implementation manner, the resolution of the display device is improved by using two OLED panels.

Solution 4: The lighting module uses three monochromatic OLED panels, such as red, green, and blue OLED panels. The three monochromatic OLED panels are placed around the light splitting element, and they are all at a 45-degree angle to the light splitting element. As shown in Figure 5d, three monochrome OLED panels emit red, green, and blue light respectively, and the light splitting device uses band-pass green and blue color beam splitters to reflect green and blue light respectively and transmit other colors of light.

The monochromatic images emitted by the three monochromatic OLED panels are fused to form a color image, which enters the light transmission unit. The resolution of the color image formed by the fusion of the monochrome images displayed by the three monochrome OLED panels is improved. Theoretically, the resolution is three times higher than that of a single display panel. Due to the use of the color light splitting element, the utilization efficiency of light can be improved, and the energy consumption of the equipment can be further reduced.

2. Light transmission unit

The light transmission unit in this embodiment may include one of the following schemes. Option 1: When the light conduction unit includes two lenses, at least one display panel is located at one focal length of one lens, and the distance between the two lenses is twice the focal length; A second predetermined distance from the location. That is, it can be considered as a 4f optical system composed of two lenses. The display panel is located at 1 times the focal length of the previous lens, the distance between the two lenses is 2 times the focal length, and the imaging plane is located at 1 times the focal length of the latter lens, as shown in the figure 6a shows.

For example, when the display panel is a self-luminous display panel, the image displayed on the display panel is transmitted to the light conduction unit through the light splitting device; when the display panel is a non-self-luminous display panel, the illumination light illuminates the display panel through the light splitter; The displayed image is imaged at a position at a second predetermined distance from the light modulating element, thereby constituting a three-dimensional virtual image of the integrated imaging display. That is, the light modulation element processes the image displayed on the display panel into a virtual object light field of a three-dimensional virtual image, and projects it to human eyes. In this embodiment, the light transmission unit images the image of the display panel near the light modulation element, so that the display panel and the light modulation element form an integrated imaging display device. This structure makes the whole system simpler, avoids the use of multiple light modulation elements, reduces the cost, and shortens the distance between the light modulation elements and human eyes, which is beneficial to improve the user's viewing angle.

At the same time, multiple display panels can use time division multiplexing to improve the resolution of the integrated imaging display, and the specific implementation is as described above. The integrated imaging display content is captured by the human eye and the camera through the beam splitter, and the real scene image of the outside world can also be captured by the human eye and the camera at the same time, realizing the augmented reality display that displays the superposition of virtual objects and real scenes.

Solution 2: when the light conduction unit includes a lens, at least one display panel is located at twice the focal length of the lens; the light modulation element is located at a third predetermined distance from the imaging position after the image is transmitted through the light conduction unit. For example, it consists of 1 lens. The display panel is located at 2 times the focal length of the lens, and the imaging surface is also located at 2 times the focal length of the lens, close to the microlens array, as shown in FIG. 6b.

It should be noted that the structure of the display device for augmented reality shown in this embodiment can be used for a display device for virtual reality, specifically, the light splitting in the display device for augmented reality shown in Fig. 6a and Fig. 6b After the mirror is changed to a mirror, the display device for augmented reality can be changed to a display device for virtual reality.

3. Light Modulation Components

Solution 1: The light modulation element is a microlens array

The image displayed on the display panel is imaged to the vicinity of the microlens array through the light transmission unit, forming an integrated imaging display system. In the integrated imaging display, many element maps are displayed on the display panel. Each element map is imaged by a corresponding microlens to form a three-dimensional object light field in space. Human eyes can perceive a three-dimensional object light field by capturing the three-dimensional object light field. Realistic 3D objects.

In this embodiment, the lens array is a planar lens array. The viewing angle in an integrated imaging display is limited by the area that each element map can display on the display panel. Usually, each microlens corresponds to a display area on the display panel. In order to prevent image overlap, the display content beyond this display area will be discarded. As shown in Figure 7, the element map cannot be displayed in the display area corresponding to the edge lens. Complete display, therefore, the number of corresponding elemental maps is limited, and the integrated image will not be observed outside the viewing angle.

Preferably, the above-mentioned problems can be solved by changing the plane lens array into a curved lens array. The display area corresponding to the edge microlens of the curved lens array will increase, and the element map can be completely displayed, and the corresponding element map can be increased. The field angle will be greatly increased, as shown in Figure 8.

Solution 2: The light modulation element is a microhole array

In the present invention, the microlens array can be replaced by a microhole array, as shown in FIG. 9 , forming an integrated imaging display system.

Both the microhole array and the microlens array have the function of making the light at a specific position be transmitted along a specific direction. The microlens array and the microhole array in the utility model can be a dynamic liquid crystal lens or a liquid crystal microhole array composed of liquid crystal elements, so that by controlling the liquid crystal element, the microlens array or the microhole array in some or all areas has a refractive effect Or there is no refraction effect, and when there is no refraction effect, it can become a transparent component, so that switching between two-dimensional and three-dimensional display can be realized, or a mixed display of two-dimensional and three-dimensional objects can be realized.

In augmented reality head-mounted display devices, displaying flat two-dimensional objects can no longer meet people's needs. If you need to display a three-dimensional object, you can wear it on both eyes and project two images with parallax to form a stereoscopic vision. In the existing 3D stereoscopic display, images with parallax are displayed on the screens corresponding to the left and right eyes, and such images will form 3D stereoscopic vision through the processing of the human brain. However, this method will cause a contradiction between focus adjustment and convergence adjustment of the human eye, and long-term wearing will make the eyes feel tired. As shown in Figure 10, when viewing a two-dimensional object on the screen, the focusing distance and convergence distance of each eye are the same, so long-term viewing will not cause eye fatigue; but in the case of viewing a three-dimensional object , because the two eyes watch the image with parallax, in order to watch a clear image, the human eye will adjust the eyes to focus on the screen, but due to the parallax, the human brain will process the image so that the three-dimensional image has a certain distance from the screen, the focus distance It is inconsistent with the convergence distance, as shown in Figure 11. At this time, the adjustment of the human eyes will make the eyes converge on the position of the three-dimensional image. Since the focus of the eyes is adjusted on the display screen, while the convergence of the eyes is focused on the spatial image point, the eyes are constantly adjusting the balance between the two. , self-adaptive, therefore, long-term wearing of this augmented reality display device will cause visual fatigue of human eyes.

In the utility model, the integrated imaging display passes through the microlens array or the microhole array, so that the reconstructed three-dimensional object is composed of many point light sources in space, and these point light sources are formed by the refraction effect of the image displayed on the display panel through the microlens array Converging, these point light sources form a real object light field distribution in three-dimensional space, just like a real object seen by human eyes, as shown in Figure 12. The integrated imaging display fully matches the focus and convergence of the human eye without causing visual fatigue caused by wearing the device for a long time. The utility model uses the principle of integrated imaging display, so that the three-dimensional objects seen by each eye are composed of real three-dimensional object light fields, just like the real scene of the outside world viewed by human eyes, so a natural three-dimensional display can be obtained , relieve the contradiction between focus and convergence adjustment, avoid visual fatigue caused by wearing a display device, and thus benefit the eye health of the viewer.

The situation of using the utility model for binocular near-eye light field display is shown in Figure 13. Since continuous convergence adjustment is provided within a certain depth of field, this binocular display can solve the problem of focus adjustment and convergence adjustment in ordinary binocular stereoscopic display. Contradictory question. It should be noted that after changing the beam splitter in the display device for augmented reality in FIG. 13 to a reflector and removing the camera, the display device for augmented reality can be changed to a display device for virtual reality.

It should be noted that the structure of the near-eye display device shown in the first embodiment of the present invention can be used for a display device for augmented reality and a display device for virtual reality.

Fig. 14 shows a display device for augmented reality according to the second embodiment of the present invention, the display device includes an illumination module, a light transmission unit and a light modulation element, wherein the light modulation element includes a microlens array or a microhole array .

Wherein, the lighting module includes: at least two display panels and a light splitting device, and the light splitting device includes a polarization beam splitter.

When the display panel is a self-illuminating display panel, for example, the display panel is an OLED display panel, the lighting module does not need an additional light source. When the display panel is a non-self-illuminating display panel, the lighting module may further include: a light source and a converging mirror.

When the display panel is a self-luminous display panel, the light modulation element is arranged in the lighting module at a position at a first predetermined distance from each display panel to form an integrated imaging display. The light modulation element processes the image displayed on the display panel into a virtual object light field, and conducts it through the light transmission unit, that is, through the relay optical system, and projects it to the human eye. Through this embodiment, human eyes can observe the virtual three-dimensional object superimposed on the real scene, thereby realizing the display of augmented reality.

When the display panel is a non-self-illuminating display panel, the illumination light illuminates the display panel through the light splitting device, and the light modulation element is directly located in the illumination module at a position at a first predetermined distance from each display panel to form an integrated imaging display. The light modulation element processes the image displayed on the display panel into a virtual object light field, and conducts it through the light transmission unit, that is, through the relay optical system, and projects it to the human eye. Through this embodiment, human eyes can observe the virtual three-dimensional object superimposed on the real scene, thereby realizing the display of augmented reality.

In this embodiment, the two display panels switch to display images at a rate greater than a predetermined refresh rate, which is much higher than the refresh rate that can be resolved by human eyes, and the displayed images are interlaced by half the pixels of the display panel in the horizontal and vertical directions. Distance, that is, the method of time division multiplexing of the display panel is used to improve the resolution. For example, if the pixel positions of the two display panels are staggered by half a pixel, the resolution can be doubled.

The device also includes a correction module, wherein the correction module includes: an image capture unit, such as a camera, a beam splitter, a correction unit, an image rendering unit, and a light source control unit.

When the display panel is a self-illuminating display panel, the light modulation element directly forms an integrated imaging display at a position in the lighting module at a first predetermined distance from the display panel, and the light modulation element processes the image displayed on the display panel into a virtual object light field, and Conducted by the light conduction unit, the light field of the virtual object enters the human eye and the image capture unit respectively through the beam splitter. The human eye and the image capture unit can simultaneously acquire images of the real scene outside. The image capture unit can receive the same image content as observed by the human eye, that is, the superposition of the 3D virtual image and the real image of the outside world to generate an augmented reality display image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit. The image is re-rendered in real time to achieve the purpose of adjusting the displayed virtual object in real time.

When the display panel is a non-self-illuminating display panel, the illumination light passes through the light splitting device to illuminate the display panel, and the light modulation element is directly located in the illumination unit at a position at a first predetermined distance from the display panel to form an integrated imaging display. The image is processed into the virtual object light field of the three-dimensional virtual image, which is transmitted through the light transmission unit, and the virtual object light field enters the human eye and the image capture unit respectively through the beam splitter. The human eye and the image capture unit can simultaneously acquire images of the real scene outside. The image capture unit can receive the same image content as observed by the human eye, that is, the superposition of the 3D virtual image and the real image of the outside world to generate an augmented reality display image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit. The image is re-rendered in real time to achieve the purpose of adjusting the displayed virtual object in real time.

The realization scheme of the lighting module in the second embodiment of the present utility model can adopt various realization schemes of the above lighting module in the first embodiment of the present utility model.

The realization scheme of the light transmission unit in the second embodiment of the present invention can adopt various realization schemes of the above light transmission unit in the first embodiment of the present invention.

It should be noted that the structure of the display device for augmented reality shown in the second embodiment of the present invention can be used for a display device for virtual reality, specifically, in the display device for augmented reality shown in Figure 14 After the beam splitter is changed to a mirror, and the camera is removed, the display device used for augmented reality can be changed to a display device used for virtual reality.

Fig. 15 shows a display device for virtual reality in the third embodiment of the present invention. The device includes: an illumination module and a light modulation element, wherein the light modulation element includes a microlens array or a microhole array.

Wherein, the lighting module includes: at least two display panels and a light splitting device; the light splitting device includes a polarization beam splitter.

When the display panel is a self-illuminating display panel, for example, the display panel is an OLED display panel, the lighting module does not need an additional light source. When the display panel is a non-self-illuminating display panel, the lighting module further includes: a light source and a converging mirror.

When the display panel is a self-illuminating display panel, the light modulation element is located in the lighting module at a position at a first predetermined distance from the display panel to form an integrated imaging display, and the light modulation element processes the image displayed on the display panel into a virtual object light field, And then projected to the human eye. Through this embodiment, human eyes can observe the virtual three-dimensional object superimposed on the real scene, thereby realizing the display of augmented reality.

When the display panel is a non-self-illuminating display panel, the illumination light passes through the light splitting device to illuminate the display panel, and the light modulation element is located in the illumination module at a position at a first predetermined distance from the display panel to form an integrated imaging display, and the light modulation element displays the display panel The image of the virtual object is processed into a light field, which is then projected to the human eye. Through this embodiment, human eyes can observe the virtual three-dimensional object superimposed on the real scene, thereby realizing the display of augmented reality.

In this embodiment, the two display panels switch to display images at a rate greater than a predetermined refresh rate, which is much higher than the refresh rate that can be resolved by human eyes, and the displayed images are interlaced by half the pixels of the display panel in the horizontal and vertical directions. Distance, that is, the method of time division multiplexing of the display panel is used to improve the resolution. For example, if the pixel positions of the two display panels are staggered by half a pixel, the resolution can be doubled.

Compared with the first embodiment and the second embodiment of the present invention above, this embodiment omits the light transmission unit, making the device more compact, and can be used in some situations where the space size is limited.

The display device also includes a correction module, wherein the correction module includes: an image capture unit, such as a camera, a beam splitter, a correction unit, an image rendering unit, and a light source control unit.

When the display panel is a self-luminous display panel, the light modulation element is located in the lighting module at a position at a first predetermined distance from the display panel to form an integrated imaging display, and the light modulation element processes the image displayed on the display panel into a virtual image of a three-dimensional virtual image. The light field of the object and the light field of the virtual object respectively enter the human eye and the image capture unit through the beam splitter. The human eye and the image capture unit can simultaneously acquire images of the real scene outside. The image capture unit can receive the same image content as observed by the human eye, that is, the superposition of the 3D virtual image and the real image of the outside world to generate an augmented reality display image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit. The image is re-rendered in real time to achieve the purpose of adjusting the displayed virtual object in real time.

When the display panel is a non-self-illuminating display panel, the illumination light illuminates the display panel through the light splitting device, and the light modulation element is directly located in the illumination module at a position at a first predetermined distance from the display panel to form an integrated imaging display. The displayed image is processed into the virtual object light field of the three-dimensional virtual image, and the virtual object light field respectively enters the human eye and the image capture unit through the beam splitter. The human eye and the image capture unit can simultaneously acquire images of the real scene outside. The image capture unit can receive the same image content as observed by the human eye, that is, the superposition of the 3D virtual image and the real image of the outside world to generate an augmented reality display image, correct the 3D virtual image through the correction unit, and correct the corrected 3D virtual image through the image rendering unit. The image is re-rendered in real time to achieve the purpose of adjusting the displayed virtual object in real time.

The solution of the lighting module in the third embodiment of the utility model can adopt various solutions of the lighting module in the first embodiment of the utility model above.

It should be noted that the structure of the display device for virtual reality shown in the third embodiment of the present invention can be used for a display device for augmented reality, specifically, in the display device for augmented reality shown in Figure 15 After the beam splitter is changed to a mirror, and the camera is removed, the display device used for augmented reality can be changed to a display device used for virtual reality.

Fig. 16 shows a display device for virtual reality in the fourth embodiment of the present invention. The display device includes a lighting module and a light modulation element; optionally, when the viewing position of the human eye is on the light path of the light emitted by the lighting module, no other components are needed; optionally, when the viewing position of the human eye is on the light path of the light emitted by the lighting module When the light path is at an angle, a reflector is needed. Figure 16 shows the setting method when the viewing position of the human eye is 90 degrees to the light path of the light emitted by the lighting module; the image from the lighting module forms a three-dimensional virtual image displayed by the integrated imaging through the light modulation element. Image; the reflector is arranged at the opposite end of the lighting module, and directs the three-dimensional virtual image to human eyes. The display device also includes a correction module, and the correction module includes: an image capture unit, an image rendering unit, and a light source control unit, the image capture unit acquires a three-dimensional virtual image from the mirror; the image rendering unit corrects the three-dimensional virtual image and renders it, And conducting through the lighting module; the light source control unit adjusts the brightness of the light emitted by the light source according to the corrected three-dimensional virtual image.

That is, on the basis of the above three embodiments of the present invention, the image capture unit and the image correction unit are removed, and the beam splitter is replaced by a reflector, so that the light field of the virtual object of the three-dimensional image information will only be captured by human eyes. The solution modified from the first embodiment is described as follows as an example. The second embodiment and the third embodiment can also obtain a similar display device for virtual reality through the same modification.

As shown in FIG. 16 , the display device includes: an illumination module, a light transmission unit, and a light modulation element, wherein the light modulation element includes a microlens array or a microhole array.

Wherein, the lighting module includes: at least two display panels and a light splitting device, and the light splitting device specifically includes a polarization beam splitter.

When the display panel is a self-illuminating display panel, for example, the display panel is an OLED display panel, the lighting module does not need an additional light source. When the display panel is a non-self-illuminating display panel, the lighting module further includes: a light source and a converging mirror.

When the display panel is a self-illuminating display panel, the image displayed on the display panel is transmitted to the light conduction unit through the light splitting device, and the light conduction unit images the image displayed on the display panel at a position at a second predetermined distance from the light modulation element, thereby forming a Integrated imaging display, the light modulation element processes the image displayed on the display panel into a virtual object light field, which is projected to the human eye. Through this embodiment, human eyes can observe virtual three-dimensional objects, thereby realizing the display of virtual reality.

When the display panel is a non-self-illuminating display panel, the illumination light illuminates the display panel through the light splitting device, and the light conduction unit images the image displayed on the display panel at a position at a second predetermined distance from the light modulation element, thereby forming an integrated imaging display, The light modulation element processes the image displayed on the display panel into a virtual object light field, which is projected to human eyes. Through this embodiment, human eyes can observe virtual three-dimensional objects, thereby realizing the display of virtual reality.

In this embodiment, the two display panels switch to display images at a rate greater than a predetermined refresh rate, which is much higher than the refresh rate that can be resolved by human eyes, and the displayed images are interlaced by half the pixels of the display panel in the horizontal and vertical directions. Distance, that is, the method of time division multiplexing of the display panel is used to improve the resolution. For example, if the pixel positions of the two display panels are staggered by half a pixel, the resolution can be doubled.

The display device includes a correction module, and the correction module includes: an image capturing unit, a mirror, an image rendering unit and a light source control unit.

When the display panel is a self-illuminating display panel, the image displayed on the display panel is transmitted to the light conduction unit through the light splitting device, and the light conduction unit images the image displayed on the display panel at a position at a second predetermined distance from the light modulation element, thereby forming a In the integrated imaging display, the light modulation element processes the image displayed on the display panel into a virtual object light field of a three-dimensional virtual image, and the displayed virtual object light field enters the human eye through a mirror. The real-time rendering is performed by the image rendering unit, so as to realize the purpose of adjusting the displayed virtual object in real time.

When the display panel is a non-self-illuminating display panel, the illumination light illuminates the display panel through the light splitting device, and the light conduction unit images the image displayed on the display panel at a position at a second predetermined distance from the light modulation element, thereby forming an integrated imaging display, The light modulation element processes the image displayed on the display panel into a virtual object light field, and the displayed virtual object light field enters human eyes through the beam splitter. The real-time rendering is performed by the image rendering unit, so as to realize the purpose of adjusting the displayed virtual object in real time. The solution of the lighting module in the fourth embodiment of the utility model can adopt various solutions of the lighting module in the first embodiment of the utility model above.

For the solution of the light transmission unit in the fourth embodiment of the present utility model, reference may be made to various solutions of the above light transmission unit in the first embodiment of the present utility model.

In another preferred embodiment of the present utility model, the basic drawing process of three-dimensional augmented reality is described in detail. As shown in FIG. In the image input processing unit, the new salient object is detected and the original object is tracked; then the new salient object is recognized; the new salient object is verified by GPS and the direction of motion; and then the new salient object is Generate a three-dimensional virtual image model of salient objects, including text or images; adjust the position of the three-dimensional virtual image in real time according to the tracking data; and calculate the contrast of the original object area; adjust the color, size, and shape of the three-dimensional virtual image according to the contrast; All the 3D virtual images are rendered into 3D images; the rendered 3D virtual images are superimposed on the real scene through the display unit.

Fig. 18 is a schematic flowchart of a three-dimensional augmented reality engine in a preferred embodiment of the present invention. The user can select an appropriate 3D enhancement mode according to needs, and the 3D enhancement mode includes the aforementioned basic drawing mode, intelligent gaze tracking mode or specific category mode. In the basic rendering mode, all salient objects in the scene will be detected and recognized, and then sent to the rendering module for adaptive rendering. In the intelligent gaze tracking mode, the system will recognize the current user's gaze direction, determine the area to be enhanced in the scene according to the gaze direction, and then detect and identify prominent objects in this area and perform adaptive enhanced display. In the specific category display mode, users can set the categories they are interested in, such as hotels, cinemas, banks, famous scenic spots, etc. in the scene, and then detect the content of the specific category in the scene according to the user's choice and perform adaptive augmented reality. In addition to the above three modes, other enhanced modes can also be designed. The utility model provides multiple three-dimensional enhancement modes for users to choose and use. The mode selection interface may adopt a menu selection or a shortcut command mode, such as setting a three-dimensional enhanced mode through a predetermined shortcut voice command.

In another preferred embodiment of the present invention, a basic rendering process of a three-dimensional light field is provided. In practical applications, two of the above-mentioned near-eye display devices (which can also be called near-eye light field displays) are required to work. The picture seen by the left eye and the right eye, so the rendering content of the two near-field displays needs to be adjusted in relation to each other. During specific implementation, the positions and angles of the content rendered by the two near-eye light-field displays can be adjusted based on the distance between the two near-eye light-field displays. As shown in Figure 19, the display device with two above-mentioned near-eye light field displays determines the three-dimensional model information that needs to be superimposed, and obtains the parameters of each near-eye light field display, wherein the obtained parameters include at least one of the following: each two-dimensional Panel pixel pitch, microlens array pitch, panel and microlens pitch, etc. The display device also acquires the distance between the left and right eye near-eye light field displays, so as to match the pupil distance of the viewer. The display device sets the positions of the left and right virtual cameras in the 3D rendering engine according to the distance between the left and right near-eye light field displays, puts the 3D model to be drawn into the 3D rendering engine, and sets the parameters according to the parameters of each near-eye light field display The parameters of the virtual camera, the set parameters include resolution and field of view, etc. Then, for the left and right virtual cameras, draw multi-angle images according to the pixel distribution under the microlens, respectively interweave and fuse the images of the left and right cameras from multiple angles, generate two elemental image arrays, and interweave and fuse The final left and right element image arrays are respectively sent to the left and right near-eye light field displays for display.

The foregoing is only a partial implementation of the utility model, and it should be pointed out that for those of ordinary skill in the art, some improvements and modifications can also be made without departing from the principle of the utility model. Retouching should also be regarded as the scope of protection of the present utility model.

Claims (21)

1. A near-eye display device, comprising a lighting module and a light modulation element: The lighting module is used to output images; The light modulation element is used for integrated imaging of the image to display a three-dimensional virtual image. 2. The near-eye display device according to claim 1, wherein the light modulation element comprises a microlens array or a microhole array. 3. The near-eye display device according to claim 2, wherein the microlens array or the microhole array is a curved microlens array or a curved microhole array. 4. The near-eye display device according to claim 2 or 3, wherein the light modulation element comprises a dynamic microlens array or a microhole array composed of liquid crystal elements. 5. The near-eye display device according to claim 1, wherein the lighting module comprises at least two display panels and a light splitting device: The at least two display panels are located on both sides of the light splitting device and form a predetermined angle with the light splitting device; Wherein, the display panel is used to display images, and the light splitting device is used to transmit the images displayed on the display panel. 6. The near-eye display device according to claim 5, wherein the at least two display panels form an angle of 45 degrees with the light splitting device. 7. The near-eye display device according to claim 5, wherein, when the display panel is a non-self-illuminating panel, the lighting module further comprises a light source and a converging mirror: The converging mirror is located between the light source and the display panel, and the light emitted by the light source illuminates the display panel through the converging mirror. 8. The near-eye display device according to claim 7, wherein the light splitting device is a polarization splitting device: The polarization beam splitting device reflects the light of the first polarization direction component collimated by the converging mirror from the light source, and transmits the light of the second polarization direction component orthogonal to it, and the transmitted and reflected polarization direction light For lighting of non-self-illuminating panels. 9. The near-eye display device according to claim 5, wherein when the display panel is a self-luminous panel, the light splitting device transmits and reflects light from the self-luminous panel. 10. The near-eye display device according to claim 5, wherein, when the display panel is a monochrome self-luminous panel, the light splitting device is a band-pass color splitting device: The band-pass color splitter reflects light of the same color as the monochromatic self-luminous panel and transmits light of other colors. 11. The near-eye display device according to claim 5, wherein the at least two display panels switch to display images at a frequency greater than a predetermined refresh rate, and the images displayed on each display panel are staggered by a predetermined ratio of pixels in the horizontal and vertical directions . 12. The near-eye display device according to claim 11, wherein when the number of display panels is n, the n display panels switch to display images at a frequency greater than n×30Hz, and the images displayed by each display panel Interleave 1/n pixels horizontally and vertically. 13. The near-eye display device according to claim 5, wherein the light modulation element is located at a position in the lighting module at a first predetermined distance from each display panel. 14. The near-eye display device according to claim 5, wherein the display device further comprises a light conduction unit, the light conduction unit includes at least one lens; the light conduction unit conducts the image displayed on the display panel to the The position at a predetermined distance from the light modulation element is used for integrated imaging of the light modulation element. 15. The near-eye display device according to claim 14, wherein when the light transmission unit includes two lenses, at least one display panel is located at one focal length of one lens, and the distance between the two lenses is twice the focal length ; the light modulation element is located at a second predetermined distance from the imaging position after the image is transmitted through the light transmission unit. 16. The near-eye display device according to claim 14, wherein when the light transmission unit comprises a lens, at least one display panel is located at twice the focal length of the lens; A third predetermined distance from the imaging position after the unit conducts. 17. The near-eye display device according to claim 1, further comprising a reflective element: The reflective element is located in the direction of the light path of the lighting module, and guides the three-dimensional virtual image to human eyes. 18. The near-eye display device according to claim 17, wherein the reflective element comprises a mirror or a beam splitter. 19. The near-eye display device according to claim 18, when the near-eye display device is a near-eye display device for augmented reality, the near-eye display device further comprises a correction module, and the reflective element is a beam splitter; The beam splitter divides the light of the three-dimensional virtual image and the real image of the outside world into two paths, which are directed to the human eye and the correction module respectively; The correction module is used to correct the three-dimensional virtual image based on the three-dimensional virtual image transmitted by the beam splitter and the real image of the outside world, and display the corrected three-dimensional virtual image through the lighting module. 20. The near-eye display device according to claim 19, the correction module comprising: An image capture unit, configured to acquire a three-dimensional virtual image from the beam splitter and a real image of the outside world; The correction unit is used to analyze the three-dimensional virtual image and the real external image, and correct the three-dimensional virtual image according to the analysis result; The image rendering unit is used for rendering the 3D virtual image obtained after correction. 21. The near-eye display device according to claim 20, wherein the correction module further comprises: The light source control unit is configured to adjust the brightness of the light emitted by the light source according to the three-dimensional virtual image obtained after correction.