[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2022117355A1 - Display device and method for operating a display device - Google Patents

Display device and method for operating a display device Download PDF

Info

Publication number
WO2022117355A1
WO2022117355A1 PCT/EP2021/082123 EP2021082123W WO2022117355A1 WO 2022117355 A1 WO2022117355 A1 WO 2022117355A1 EP 2021082123 W EP2021082123 W EP 2021082123W WO 2022117355 A1 WO2022117355 A1 WO 2022117355A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
light
eye
receiver unit
unit
Prior art date
Application number
PCT/EP2021/082123
Other languages
French (fr)
Inventor
Jens Hofrichter
Corneliu-Mihai Tobescu
Original Assignee
Ams International Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ams International Ag filed Critical Ams International Ag
Publication of WO2022117355A1 publication Critical patent/WO2022117355A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B27/0172Head mounted characterised by optical features
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B2027/0178Eyeglass type

Definitions

  • the present disclosure relates to a display device , such as an augmented or mixed reality device , and to a method for operating such a display device .
  • AR and MR devices feature a partially transparent display that allows the reality to be viewed with additional spatially related virtual content superimposed on the view of reality .
  • Typical examples of these are AR or MR glasses or windshields of vehicles featuring a heads-up display .
  • tracking eye movements of the user are essential .
  • conventional systems employ additional sensors , e . g . cameras , in order to determine a viewing direction of the eyes of the user .
  • additional sensors e . g . cameras
  • Such additional sensors occupy additional space , in turn causing bulkiness of existing solutions and elevated cost , in addition to a signi ficantly increased energy budget , which is typically a critically limited resource .
  • transmitters in in existing solutions typically employ 2D-arrays of micro-LEDs or micro VECSELs , which require additional proj ection optics , such as prisms or mirrors , for displaying the content on a glass substrate , for instance .
  • proj ection optics such as prisms or mirrors
  • Such elements not only cause further bulkiness of existing solutions but also introduce signi ficant optical losses , in turn increasing the power budget . It is an obj ect to provide an improved concept of a display device and of a method for operating a display device that overcomes limitations of existing solutions .
  • the improved concept is based on the idea of providing a display device having a waveguide coupled bidirectional proj ection element , which is employed for directing visual content to the eye of a user in addition to receiving information from the eye for eye tracking and/or iris recognition purposes , for instance .
  • external proj ection elements such as mirrors or prisms , as well as separate detection systems , e . g . for eye tracking applications , become obsolete .
  • a display device comprises a transparent substrate , and a display area on or within the substrate comprising a bidirectional proj ection element , which is configured to direct a light signal to an eye of a user and to receive light that is reflected from the eye .
  • the display area is at least partially transparent and is configured to enable viewing of an ambient environment through the display area .
  • the display device further comprises a transmitter unit that is configured to generate the light signal based on an image signal .
  • the display device further comprises a receiver unit that is configured to generate a detection signal based on the light received by the bidirectional proj ection element .
  • the display device further comprises an optical waveguide structure that couples the transmitter unit and the receiver unit to the bidirectional projection element, and a processer that is configured to provide the image signal to the transmitter unit and to process the detection signal from the receiver unit .
  • the transparent substrate is, for example, a glass substrate such as an eyeglass lens or a window.
  • a section of the glass substrate acts as a partially transparent display for displaying content, hence realizing an AR or MR experience for a user looking through the glass substrate.
  • said section comprises a projection element for directing light to the eye of the user.
  • the projection element can be waveguide based, such as a single or multi-layered waveguide display.
  • the projection element can be of diffractive or of holographic type.
  • the projection element is or comprises a diffractive grating or a holographic optical element.
  • the projection element can be characterized by a patterning within a core region of the waveguide.
  • the projection element can be a patterned or structured region of waveguides of the waveguide structure.
  • the projection element can be operated in a bidirectional manner.
  • the bidirectional projection element in addition to having transmitting capabilities for directing light to an eye of the user, i.e. displaying virtual content within the display area, wherein the light is provided to the projection element from a transmitter unit via the waveguide structure, the bidirectional projection element is further configured to receive optical signals.
  • the bidirectional projection element receives optical signals from the eye of the user and provides these optical signals to a receiver unit via the waveguide structure. Therefore, the bidirectional projection element can be understood as a coupling element that couples light from the waveguide structure to free space towards the eye of the user, and light from said free space back into the waveguide structure .
  • the waveguide structure comprises an optical waveguide that couples the transmitter unit and the receiver unit to the bidirectional proj ection element .
  • the optical waveguide of the waveguide structure directly couples said elements without any free space path or further optical elements , such as mirrors or prisms , in between .
  • the waveguide structure can comprise a waveguide beam splitter or a waveguide-based optical circulator for ensuring that light from the transmitter unit is exclusively directed to the bidirectional proj ection element , and hence to the eye of the user, and light received by the bidirectional proj ection element is predominantly directed to the receiver unit .
  • the transmitter unit can be understood as a converter unit that converts a digital image signal received from a processor into a light signal and emits this light signal into the waveguide structure and towards the bidirectional proj ection element using light sources .
  • the receiver element can analogously be understood as a converter unit that converts light received from the bidirectional proj ection element into a detection signal which is provided to the processor by means of optical detectors .
  • the processor can comprise a memory element or it can be coupled to an external memory for accessing the image signal for providing the latter to the transmitter unit .
  • the processor is further configured to process the detection signal .
  • the processor is configured to analyze the detection signal by performing an image recognition process .
  • the transmitter unit and the receiver unit are comprised by a transceiver unit
  • the optical waveguide structure is a bidirectional optical waveguide structure coupling the transceiver unit to the bidirectional proj ection element .
  • the transmitter unit and the receiver unit can be integrated in an electro-optical transceiver unit .
  • This way only a single waveguide-coupled integrated circuit device is required, hence easing the complexity of the waveguide structure and leading to less estate required for electronic components , the latter particularly for wearable devices constituting a signi ficant advantage in terms of aesthetics .
  • the transmitter unit comprises an array, in particular a one-dimensional array, of light sources .
  • the transmitter unit In order to be able to display information within the display area of the device , light emitters comprised by the transmitter unit establish the link between electronic image signals received from the processor to optical signals that can be directed via the waveguide structure and the proj ection element to the eye of the user .
  • the transmitter unit can comprise an array of micro-LEDs or micro-VCSELs configured to emit light into the optical waveguide structure towards the proj ection element .
  • the transmitter unit comprises a one-dimensional array of emitters , wherein each emitter addresses one column or row of pixels of the display area defined by a structure of the proj ection element , which reali zes a waveguide display, for instance .
  • the receiver unit comprises an array, in particular a one-dimensional array, of light detectors .
  • the receiver unit in these embodiments comprise lights detectors for establishing the link between light signals that are captured by the proj ection element and provided to the transmitter unit via the waveguide structure and electronic detection signals that are provided to the processor for further processing .
  • the receiver unit can comprise an array of micro photodiodes configured to convert light signals received via the optical waveguide structure into electronic detection signals .
  • the receiver unit comprises a onedimensional array of detectors , wherein each detector addresses one column or row of pixels of a detection some range of the display area, or alternatively of the entire display area .
  • the bidirectional proj ection element comprises a di f fractive grating or a holographic optical element .
  • a near-eye display is reali zed via optical planar waveguides with di f fraction gratings for coupling light from the waveguide structure to the eye of the user, and for wavelength selection and wave front reshaping, for example .
  • the bill directional proj ection element can be a holographic optical element , which simultaneously performs the optical functions of a mirror and a lens .
  • both types of elements can be designed to operate in a bidirectional manner such that the detection of light impinging on the proj ection element is enabled, hence removing the necessity for a dedicated sensor, e . g . , a camera, for eye tracking purposes , for instance .
  • the transmitter unit is configured to emit light in the visible domain and in the infrared, in particular the near infrared, domain .
  • the light directed to the eye comprises visual content in the visible domain and an infrared, in particular a near infrared, signal .
  • the transmitter unit is configured to display a visible image on the display area and to perform an eyetracking or iris-recognition process , for example , in the infrared domain .
  • the transmitter unit can be configured to emit an infrared signal via the waveguide structure and the proj ection element to the eye of the user such that from a reflection of this infrared signal , a momentary condition or parameter of the eye can be inferred .
  • the transmitter unit transmits a pulse of infrared light at a certain frequency, e . g . , a frequency that corresponds to the typical reaction time of the human eye of around 60 ms , in order to track a position of the eye ' s pupil .
  • the receiver unit is configured to generate the detection signal based on infrared, in particular near infrared, light .
  • the receiver unit can comprise detectors that are predominantly sensitive in the infrared domain, such that unwanted background reflections in the visible domain do not falsi fy the measurement and thus leads to an increased accuracy .
  • the transmitter unit and the receiver unit are coupled to the optical waveguide structure via vertical coupling or via lateral butt-coupling .
  • Directly coupling optical waveguides of the waveguide structure via vertical coupling or lateral butt-coupling can enable an optical ef ficiency that is nearly lossless .
  • both a light emitter array and a sensor array in these embodiments are directly coupled to the waveguide structure without any further optical elements , such as mirrors or prisms , or free space paths in between .
  • This prevents the introduction of signi ficant loss channels which overall improves the power budget of the device , particularly of signi ficance for battery-powered devices such as smart eyeglasses .
  • the processor is further configured to determine from the detection signal at least one of : a direction of view and/or an accommodation of the eye , an identity of the user via iris recognition, and a cardiac cycle of the user .
  • the light that is reflected from the eye can be reflected from a retina or an iris of the eye , such that the light received within the display area by means of the proj ection element can be analyzed in terms of a viewing direction and or in terms of a focusing of the user' s eye , or it can be used to determine the identity of the user via iris recognition .
  • reflection from blood vessels within the retina can give information about a heartrate of the user .
  • the receiver unit is configured to generate the detection signal based on light that is received by the bidirectional proj ection element within a partial area of the display area .
  • the receiver unit can comprise optical detectors that are configured to only read out this sub range of the display area .
  • providing only a limited detection window can save a signi ficant amount of energy, and thus battery power i f no high-resolution image of the reflected light is required .
  • the display device further comprises a spectacle frame , wherein the transparent substrate is an eyeglass lens supporting by the spectacle frame .
  • the spectacle frame acts as a housing for the transmitter unit , the receiver unit and the processor .
  • the display device is a wearable device such as AR or MR smart eyeglasses , wherein the display area is reali zed on lenses of the eyeglass .
  • the spectacle frame acts as a support for the eyeglass lenses but can also provide housing for the electronic component such as the transmitter unit , the receiver unit , and the processor .
  • the display device can further comprise an energy source such as a battery, and a memory element both likewise integrated into the spectacle frame .
  • an optical path between the eye and the bidirectional proj ection element is the only free space optical path .
  • free space optical paths typically introduce a signi ficant loss channel and are thus avoided in these embodiments except for the obvious path between the display and the eye of the user .
  • the proj ection element and the transmitter and receiver units are directly coupled to each other via the waveguide structure and without any free-space paths or other optical elements , such as mirrors , prisms , couplers or lenses .
  • the display device is free of a dedicated detection system for monitoring a direction of view of the eye .
  • proj ection element also for detection purposes removes the need for equipping the display device with a camera, for example , for eye-tracking or authentication purposes , for instance .
  • the improved concept of these embodiments allows for these processes to be performed by analyzing light that is reflected back onto the proj ection element and coupled back into the waveguide structure .
  • FIG. 1 Further possible applications for a display device according to the improved concept besides smart eyeglasses are fields , where a content is being displayed on a transparent screen while requiring sensing functionality . Further examples for such applications are : smart windows for dynamic displays , 3- D advertisement screens , TV screens with embedded sensing, heads up displays with integrated sensing functionality, and virtual cockpits in automotive with integrated sensing, e . g . for eye-tracking or fatigue detection purposes , for instance .
  • the aforementioned obj ect is further achieved by a method for operating a display device .
  • the method comprises providing a display area on or within a transparent substrate of a display device , wherein the display area comprises a bidirectional proj ection element .
  • the method further comprises providing, by means of a processor, an image signal to a transmitter unit , generating, by means of the transmitter unit , a light signal based on the image signal , and providing by means of an optical waveguide structure , the light signal to the bidirectional proj ection element .
  • the method according to the improved concept further comprises directing, by means of the bidirectional proj ection element , the light signal to an eye of a user, and receiving light that is reflected from the eye .
  • the method further comprises providing, by means of the optical waveguide structure , the received light to a receive unit , and generating, by means of the receiver unit , a detection signal based on the received light .
  • the method further comprises processing, by means of the processor, the detection signal . Further embodiments of the method become apparent to the skilled reader from the embodiments of the display device as described above .
  • Figure 1 shows a schematic view of an exemplary embodiment of a display device according to the improved concept ;
  • Figures 2 and 3 show further exemplary embodiments of a display device according to the improved concept ;
  • Figures 4 and 5 illustrate di f ferent configurations of the transmitter and receiver units and of the waveguide structure ;
  • Figures 6 to 10b illustrate various embodiments of waveguide coupling employed in a display device according to the improved concept .
  • FIG. 1 shows a schematic view of an exemplary embodiment of a display device 1 according to the improved concept .
  • the display device 1 comprises a transparent substrate 10 , which is a glass or a polymer substrate , for instance .
  • Transparent in this context means that the substrate is transmissive across the visible wavelength range of electromagnetic radiation .
  • Typical material choices for the transparent substrate in the domain of wearable devices are optical crown glass , such borosilicate or BK7 , and allyl diglycol carbonate , also known as ADC or CR-39 .
  • the transparent substrate can be a window or more speci fically a windshield made of laminated safety glass .
  • a waveguide structure 13 is arranged comprising optical waveguides 13a, having waveguide cores , and a cladding 13b .
  • the waveguide structure 13 is configured to guide electromagnetic waves in the optical spectrum, in particular in the visible domain and optionally in a subrange of the infrared domain, e . g . the near infrared domain, along a main direction of extension of the waveguide structure 13 .
  • Optical waveguides themselves are a well-known concept and are not further detailed throughout this disclosure .
  • the bidirectional proj ection element 12 is arranged within the cladding 13b and is coupled, in particular directly coupled, to the optical waveguides 13a .
  • the bidirectional proj ection element 12 is configured to direct light received from the optical waveguides 13a to an eye 2 of a user of the display device 1 .
  • the bidirectional proj ection element 12 defines a display area 11 , in which visual content , e . g . virtual augmented reality content , is formed from the perspective of the user .
  • the bidirectional proj ection element 12 is a di f fractive optical element , in which each row or column is addressed by one optical waveguide 13a of the waveguide structure 13.
  • the bidirectional projection element 12 is a holographic optical element. As illustrated, the bidirectional projection element 12 can be waveguide based. For example, the bidirectional projection element 12 constitutes an extension of the optical waveguides 13, wherein a core region is patterned or structured to form the bidirectional projection element 12.
  • the bidirectional projection element 12 is further configured to receive light from the eye 2 of the user and couple this light to the optical waveguides 13a of the optical waveguide structure 13.
  • the bidirectional projection element 12 can be understood as a bidirectional optical coupling element such as a Bragg grating coupling element.
  • the display area 11 is formed by a waveguide display comprising optical waveguides 13a and a bidirectional projection element 12.
  • the optical waveguide structure 13 and the projection element 12 can be arranged within a transparent substrate 10.
  • the light received by the projection element 12 is light that is emitted by the transmitter unit 14a, guided via the waveguide structure 13 and the projection element 12 to the user's eye 2, and reflected back, e.g. from the retina or the iris, towards the projection element 12.
  • the transmitter unit 14a can be configured to emit visible light for forming an image, e.g. a virtual content image, on the display surface 11, and to emit infrared light, in particular near-infrared light, that is directed to the user's eye 2, reflected back to the bidirectional projection element 12 and guided to the receiver unit 14b for detection .
  • This signal can be processed by means of the processor 15 , for example , for determining a direction of view, an accommodation of the eye 2 , an identity of the user via iris recognition, and/or a cardiac cycle of the user .
  • the transmitter unit 14a can be configured to emit pulses of infrared light , e . g . with a period of around 60 ms constituting the typical reaction time of a human eye .
  • the display area 11 comprising the proj ection element 12 is at least partially transparent with respect to the visible domain, e . g . a wavelength between 400 nm and 800 nm .
  • the optical waveguides 13a, the cladding 13b and the proj ection element 12 are at least partially transparent .
  • the display device 1 further comprises a transceiver unit 14 comprising a transmitter unit 14a and a receiver unit 14b .
  • the transceiver unit 14 is coupled to , in particular directly coupled to , the waveguide structure 13 .
  • the transceiver unit 14 comprises a silicon chip or substrate having an integrated circuit for controlling the transmitter unit 14a and the receiver unit 14b .
  • Circuitry elements of the transceiver unit 14 can be coupled to the transmitter unit 14a and the receiver unit 14b via interconnection elements , such as wire bonds or TSVs in combination with solder bumps , for instance .
  • the transmitter unit 14a for example comprises light emitters , such as micro-LEDs or micro VCSELs , configured to illuminate pixels of the display area 11 .
  • the transmitter unit 14a comprises an array, e . g . a one- dimensional array, of light emitters that are coupled to the bidirectional proj ection element 12 via optical waveguides 13a of the waveguide structure 13 .
  • the transmitter unit 14a can be regarded a converter unit for converting a digital image signal into a visible light signal that is directed to the eye 2 of the user .
  • the receiver unit 14b for example comprises light detectors , such as micro photodiodes , configured to detect light that is received by the bidirectional proj ection element 12 and guided to the receiver unit 14b via the optical waveguide structure 13 .
  • the transmitter unit 14a comprises an array, e . g . a onedimensional array, of light detectors that are coupled to the bidirectional proj ection element 12 via optical waveguides 13a of the waveguide structure 13 .
  • the receiver unit 14b can be regarded a converter unit for converting a visible light signal received from the eye 2 of the user into a digital detection signal .
  • the receiver unit 14b can be configured to detect light captured by the bidirectional proj ection element 12 within a predefined subrange of the display area 11 .
  • the receiver unit 14b receives light captured by the bidirectional proj ection element 12 within a 200 to 200 pixels subrange .
  • the display device 1 further comprises a processor 15 that is coupled to the transceiver unit 14 , and is configured to control the transmitter unit 14a via providing the aforementioned digital image signal representing an image to be displayed on the display area 11 .
  • the processor 15 is further configured to receive from the receiver unit 14b the digital detection signal and process the signal according to a predefined process .
  • the processor 15 is configured to apply an eye-tracking or iris authentication process on the digital detection signal .
  • FIGS 2 and 3 show a further exemplary embodiments of a display device 1 , which in these cases are configured as a wearable display device 1 , such as smart eyeglasses .
  • the display device 1 comprises a spectacle frame 16a supporting the eyeglass lenses 16b, which each act as a transparent substrate 10 according to the improved concept .
  • the bidirectional proj ection element 12 defining the display area 11 is arranged .
  • the bidirectional proj ection element 12 is coupled to the transmitter unit 14a and to the receiver unit 14b via the waveguide structure 13 .
  • the transmitter unit 14a and the receiver unit 14b are designed as separate units .
  • the spectacle frame 16a can comprise cavities for providing housing for the transmitter and receiver units 14a, 14b, having separate units can be desirable in order to arrange both components within the spectacle frame 16a in an inconspicuous manner .
  • the aforementioned transceiver unit 14 includes the transmitter and receiver units 14a, 14b in a single unit .
  • This uses the concept of the optical waveguides 13a operating in a bidirectional manner .
  • the optical waveguides 13a transmit light signals from the transmitter unit 14a to the bidirectional proj ection element 12 and from the bidirectional proj ection element 12 to the receiver unit 14b .
  • ef fectively only hal f the amount of optical waveguides 13a of the waveguide structure 13 is required .
  • FIG. 4 illustrates a first configuration of the transmitter and receiver units 14a, 14b and of the waveguide structure 13 of a display device 1 according to the improved concept .
  • a transceiver 14 comprising a transceiver integrated circuit , e . g . arranged on a silicon chip or substrate , comprises an integrated receiver unit 14b that is coupled to a receiving portion of the waveguide structure 13 .
  • the transmitter unit 14a comprises a dedicated chip on an optical substrate , e . g . an A12O3 , GaAs , InP, which is electrically connected to a driver chip or IC, which preferably is made from silicon .
  • the transmitter unit 14a is coupled to the transceiver 14 via interconnects such as wire bonds or TSVs and bumps .
  • the transmitter unit 14a is coupled to a transmitting portion of the waveguide structure 13 .
  • the transmitter unit 14a comprises an array of light emitters , such as a MEMS array of lasers .
  • the receiver unit 14b comprises an array of lights detectors , such as mems array of micro photodiodes .
  • This first configuration shown in figure 4 corresponds to that illustrated in the embodiment of the display device 1 of figure 2 , for example , in which the waveguide structure 13 is operated in a unidirectional manner .
  • the second configuration of the transmitter and receiver units 14a, 14b and of the waveguide structure 13 of a display device 1 illustrated in figure 5 in contrast corresponds to that illustrated in the embodiment of the display device 1 of figure 3 , for example .
  • the waveguide structure 13 is operated in a bidirectional manner .
  • both the transmitter unit 14a and the receiver unit 14b are arranged on a common substrate , which may be an A12O3 , GaAs , InP, which is electrically connected to a driver chip or IC, which preferably is made from silicon .
  • the transmitter unit 14a and the receiver unit 14b are coupled to the transceiver 14 comprising a transceiver integrated circuit via interconnects such as wire bonds or TSVs and bumps .
  • Figure 6 illustrates a waveguide splitter or combiner particularly used in embodiments of the display device 1 , in which the waveguide structure 13 is operated in a bidirectional manner .
  • This way cross talk between the light that this to be guided from the transmitter unit 14a to the bidirectional proj ection element 12 and light that is to be guided from the bidirectional proj ection element 12 to the receiver unit 14b is minimi zed .
  • an optical circulator in particular a waveguide-based optical circulator, can be employed .
  • the propagation direction of light as illustrated by means of the arrows in figure 6 .
  • Figure 7 schematically illustrates a first waveguide coupling configuration employed in a display device 1 according to the improved concept .
  • figure 7 illustrates vertical coupling of the waveguide structure 13 to the transmitter unit 14a .
  • the transmitter unit 14a is shown to comprise a single emission layer 142 buried within a cladding 141 and arranged on a substrate 140
  • the waveguide structure comprises a single waveguide having a waveguide core 13a inside the cladding 13b coupled to the emission layer 142 .
  • the emission layer 142 reali zes a VCSEL or a micro-LED .
  • the optical waveguide core 13a is directly vertically coupled to the transmitter engine 14a .
  • FIG 8 schematically illustrates a second waveguide coupling configuration, in which the optical waveguide core 13a is directly coupled to the transmitter engine 14a via lateral butt-coupling .
  • the transmitter unit 14a is shown to comprise a single emission layer 142 that reali zes a laterally emitting light source , e . g . an LED, a DFB laser, or a DBR laser .
  • FIG 9 schematically illustrates the first waveguide coupling configuration for the receiver unit 14b .
  • the receiver unit 14b is shown to comprise a single detection layer 143 within a substrate 140 , e . g . a silicon substrate , and covered by an oxide cladding 141 , for instance .
  • the receiver unit 14 B is directly coupled to the waveguide structure 13 via vertical coupling in this configuration .
  • Figure 10a schematically illustrates a cross-sectional view of a second waveguide coupling configuration for the receiver unit 14b .
  • the receiver unit 14b is arranged with respect to the waveguide structure 13 in a lateral manner .
  • the dashed line illustrates the light path .
  • the waveguide cladding 13 B can be at least partially removed in the region of the receiver unit 14b to enhance the detection ef ficiency .
  • the detection layer 143 is arranged within a substrate 140 and covered by a silicon dioxide backend of line cladding 141 .
  • Figure 10b schematically illustrates a top view of the second waveguide coupling configuration of figure 10b .
  • the waveguide core 13a can have a tapered profile in the detection region, i . e . in the region of the receiver unit 14b .
  • the coupling configuration illustrated in figures 6 to 10b merely serve illustration purposes .
  • Actual embodiments of the transmitter unit 14a and the receiver unit 14b can comprise a plurality of light emitters and light detectors .
  • the transmitter unit 14a and the receiver unit 14b comprise arrays of light emitters and light detectors , respectively .
  • the waveguide structure 13 coupled to the transmitter unit 14a and to the receiver unit 14b comprises a plurality of optical waveguides , for example matching an amount of the plurality of light emitters and detectors .
  • the embodiments of the display device 1 and embodiments of its operation method shown in the figures represent exemplary embodiments , therefore they do not constitute a complete list of all embodiments according to the improved concept .
  • Actual display devices may vary from the embodiments shown in terms of additional components , shape and configuration, for instance .
  • features shown in the various figures may be combined with each other and hence form additional embodiments according to the improved concept .

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Optical Integrated Circuits (AREA)

Abstract

A display device (1) comprises a transparent substrate (10), a display area (11) on or within the substrate (10) comprising a bidirectional projection element (12) configured to direct a light signal to an eye (2) of a user and to receive light that is reflected from the eye (2). The display area (11) is at least partially transparent and is configured to enable viewing of an ambient environment (3) through the display area (11). The display device (1) further comprises a transmitter unit (14a) configured to generate the light signal based on an image signal, a receiver unit (14b) configured to generate a detection signal based on the light received by the bidirectional projection element (12), an optical waveguide structure (13) coupling the transmitter unit (14a) and the receiver unit (14b) to the bidirectional projection element (12), and a processor (15) configured to provide the image signal to the transmitter unit (14a) and to process the detection signal from the receiver unit (14b).

Description

Description
DISPLAY DEVICE AND METHOD FOR OPERATING A DISPLAY DEVICE
The present disclosure relates to a display device , such as an augmented or mixed reality device , and to a method for operating such a display device .
State of the art augmented reality (AR) and mixed reality (MR) devices feature a partially transparent display that allows the reality to be viewed with additional spatially related virtual content superimposed on the view of reality . Typical examples of these are AR or MR glasses or windshields of vehicles featuring a heads-up display . For a reliable AR or MR experiences , tracking eye movements of the user are essential . To this end, conventional systems employ additional sensors , e . g . cameras , in order to determine a viewing direction of the eyes of the user . Such additional sensors , however, occupy additional space , in turn causing bulkiness of existing solutions and elevated cost , in addition to a signi ficantly increased energy budget , which is typically a critically limited resource .
Similarly, transmitters in in existing solutions typically employ 2D-arrays of micro-LEDs or micro VECSELs , which require additional proj ection optics , such as prisms or mirrors , for displaying the content on a glass substrate , for instance . Such elements not only cause further bulkiness of existing solutions but also introduce signi ficant optical losses , in turn increasing the power budget . It is an obj ect to provide an improved concept of a display device and of a method for operating a display device that overcomes limitations of existing solutions .
This obj ect is achieved with the subj ect-matter of the independent claims . Further embodiments are the subj ectmatter of the dependent claims .
The improved concept is based on the idea of providing a display device having a waveguide coupled bidirectional proj ection element , which is employed for directing visual content to the eye of a user in addition to receiving information from the eye for eye tracking and/or iris recognition purposes , for instance . In this manner, external proj ection elements , such as mirrors or prisms , as well as separate detection systems , e . g . for eye tracking applications , become obsolete .
A display device according to the improved concept comprises a transparent substrate , and a display area on or within the substrate comprising a bidirectional proj ection element , which is configured to direct a light signal to an eye of a user and to receive light that is reflected from the eye . Therein, the display area is at least partially transparent and is configured to enable viewing of an ambient environment through the display area . The display device further comprises a transmitter unit that is configured to generate the light signal based on an image signal . The display device further comprises a receiver unit that is configured to generate a detection signal based on the light received by the bidirectional proj ection element . The display device further comprises an optical waveguide structure that couples the transmitter unit and the receiver unit to the bidirectional projection element, and a processer that is configured to provide the image signal to the transmitter unit and to process the detection signal from the receiver unit .
The transparent substrate is, for example, a glass substrate such as an eyeglass lens or a window. Therein, at least a section of the glass substrate acts as a partially transparent display for displaying content, hence realizing an AR or MR experience for a user looking through the glass substrate. To this end, said section comprises a projection element for directing light to the eye of the user. The projection element can be waveguide based, such as a single or multi-layered waveguide display. The projection element can be of diffractive or of holographic type. For example, the projection element is or comprises a diffractive grating or a holographic optical element. The projection element can be characterized by a patterning within a core region of the waveguide. The projection element can be a patterned or structured region of waveguides of the waveguide structure.
According to the improved concept, the projection element can be operated in a bidirectional manner. In other words, in addition to having transmitting capabilities for directing light to an eye of the user, i.e. displaying virtual content within the display area, wherein the light is provided to the projection element from a transmitter unit via the waveguide structure, the bidirectional projection element is further configured to receive optical signals. For example, the bidirectional projection element receives optical signals from the eye of the user and provides these optical signals to a receiver unit via the waveguide structure. Therefore, the bidirectional projection element can be understood as a coupling element that couples light from the waveguide structure to free space towards the eye of the user, and light from said free space back into the waveguide structure .
The waveguide structure comprises an optical waveguide that couples the transmitter unit and the receiver unit to the bidirectional proj ection element . In particular, the optical waveguide of the waveguide structure directly couples said elements without any free space path or further optical elements , such as mirrors or prisms , in between . The waveguide structure can comprise a waveguide beam splitter or a waveguide-based optical circulator for ensuring that light from the transmitter unit is exclusively directed to the bidirectional proj ection element , and hence to the eye of the user, and light received by the bidirectional proj ection element is predominantly directed to the receiver unit .
The transmitter unit can be understood as a converter unit that converts a digital image signal received from a processor into a light signal and emits this light signal into the waveguide structure and towards the bidirectional proj ection element using light sources . The receiver element can analogously be understood as a converter unit that converts light received from the bidirectional proj ection element into a detection signal which is provided to the processor by means of optical detectors .
The processor can comprise a memory element or it can be coupled to an external memory for accessing the image signal for providing the latter to the transmitter unit . The processor is further configured to process the detection signal . For example , the processor is configured to analyze the detection signal by performing an image recognition process .
In some embodiments , the transmitter unit and the receiver unit are comprised by a transceiver unit , and the optical waveguide structure is a bidirectional optical waveguide structure coupling the transceiver unit to the bidirectional proj ection element .
The transmitter unit and the receiver unit can be integrated in an electro-optical transceiver unit . This way, only a single waveguide-coupled integrated circuit device is required, hence easing the complexity of the waveguide structure and leading to less estate required for electronic components , the latter particularly for wearable devices constituting a signi ficant advantage in terms of aesthetics .
In some embodiments , the transmitter unit comprises an array, in particular a one-dimensional array, of light sources .
In order to be able to display information within the display area of the device , light emitters comprised by the transmitter unit establish the link between electronic image signals received from the processor to optical signals that can be directed via the waveguide structure and the proj ection element to the eye of the user . To this end, the transmitter unit can comprise an array of micro-LEDs or micro-VCSELs configured to emit light into the optical waveguide structure towards the proj ection element . For example , the transmitter unit comprises a one-dimensional array of emitters , wherein each emitter addresses one column or row of pixels of the display area defined by a structure of the proj ection element , which reali zes a waveguide display, for instance .
In some embodiments , the receiver unit comprises an array, in particular a one-dimensional array, of light detectors .
Analogous to the transmitter unit , the receiver unit in these embodiments comprise lights detectors for establishing the link between light signals that are captured by the proj ection element and provided to the transmitter unit via the waveguide structure and electronic detection signals that are provided to the processor for further processing . To this end, the receiver unit can comprise an array of micro photodiodes configured to convert light signals received via the optical waveguide structure into electronic detection signals . For example , the receiver unit comprises a onedimensional array of detectors , wherein each detector addresses one column or row of pixels of a detection some range of the display area, or alternatively of the entire display area .
In some embodiments , the bidirectional proj ection element comprises a di f fractive grating or a holographic optical element .
For example , a near-eye display is reali zed via optical planar waveguides with di f fraction gratings for coupling light from the waveguide structure to the eye of the user, and for wavelength selection and wave front reshaping, for example . Alternatively, the bill directional proj ection element can be a holographic optical element , which simultaneously performs the optical functions of a mirror and a lens . According to the improved concept , both types of elements can be designed to operate in a bidirectional manner such that the detection of light impinging on the proj ection element is enabled, hence removing the necessity for a dedicated sensor, e . g . , a camera, for eye tracking purposes , for instance .
In some embodiments , the transmitter unit is configured to emit light in the visible domain and in the infrared, in particular the near infrared, domain .
In some embodiments , the light directed to the eye comprises visual content in the visible domain and an infrared, in particular a near infrared, signal .
For example , the transmitter unit is configured to display a visible image on the display area and to perform an eyetracking or iris-recognition process , for example , in the infrared domain . In these embodiments , the transmitter unit can be configured to emit an infrared signal via the waveguide structure and the proj ection element to the eye of the user such that from a reflection of this infrared signal , a momentary condition or parameter of the eye can be inferred . For example , the transmitter unit transmits a pulse of infrared light at a certain frequency, e . g . , a frequency that corresponds to the typical reaction time of the human eye of around 60 ms , in order to track a position of the eye ' s pupil .
In some embodiments , the receiver unit is configured to generate the detection signal based on infrared, in particular near infrared, light . In embodiments , in which a detection is performed in the infrared, e . g . , for eye-tracking or authentication purposes , the receiver unit can comprise detectors that are predominantly sensitive in the infrared domain, such that unwanted background reflections in the visible domain do not falsi fy the measurement and thus leads to an increased accuracy .
In some embodiments , the transmitter unit and the receiver unit are coupled to the optical waveguide structure via vertical coupling or via lateral butt-coupling .
Directly coupling optical waveguides of the waveguide structure via vertical coupling or lateral butt-coupling can enable an optical ef ficiency that is nearly lossless . Hence , both a light emitter array and a sensor array in these embodiments are directly coupled to the waveguide structure without any further optical elements , such as mirrors or prisms , or free space paths in between . This prevents the introduction of signi ficant loss channels , which overall improves the power budget of the device , particularly of signi ficance for battery-powered devices such as smart eyeglasses .
In some embodiments , the processor is further configured to determine from the detection signal at least one of : a direction of view and/or an accommodation of the eye , an identity of the user via iris recognition, and a cardiac cycle of the user .
For example , the light that is reflected from the eye can be reflected from a retina or an iris of the eye , such that the light received within the display area by means of the proj ection element can be analyzed in terms of a viewing direction and or in terms of a focusing of the user' s eye , or it can be used to determine the identity of the user via iris recognition . Moreover, reflection from blood vessels within the retina can give information about a heartrate of the user .
In some embodiments , the receiver unit is configured to generate the detection signal based on light that is received by the bidirectional proj ection element within a partial area of the display area .
In particular for eye-tracking purposes , it can be shown that a subrange of a few hundred pixels , e . g . , 400 x 400 pixels or even 200 x 200 pixels can be suf ficient for accurately determining a viewing direction of the eye . Hence , the receiver unit can comprise optical detectors that are configured to only read out this sub range of the display area . In contrast to embodiments in which light is received by the proj ection element across the entire display area, providing only a limited detection window can save a signi ficant amount of energy, and thus battery power i f no high-resolution image of the reflected light is required .
In some embodiments , the display device further comprises a spectacle frame , wherein the transparent substrate is an eyeglass lens supporting by the spectacle frame .
In some further embodiments , the spectacle frame acts as a housing for the transmitter unit , the receiver unit and the processor . For example , the display device is a wearable device such as AR or MR smart eyeglasses , wherein the display area is reali zed on lenses of the eyeglass . The spectacle frame acts as a support for the eyeglass lenses but can also provide housing for the electronic component such as the transmitter unit , the receiver unit , and the processor . Optionally, the display device can further comprise an energy source such as a battery, and a memory element both likewise integrated into the spectacle frame .
In some embodiments , an optical path between the eye and the bidirectional proj ection element is the only free space optical path .
As mentioned above , free space optical paths typically introduce a signi ficant loss channel and are thus avoided in these embodiments except for the obvious path between the display and the eye of the user . In particular, the proj ection element and the transmitter and receiver units are directly coupled to each other via the waveguide structure and without any free-space paths or other optical elements , such as mirrors , prisms , couplers or lenses .
In some embodiments , the display device is free of a dedicated detection system for monitoring a direction of view of the eye .
Using the proj ection element also for detection purposes removes the need for equipping the display device with a camera, for example , for eye-tracking or authentication purposes , for instance . The improved concept of these embodiments allows for these processes to be performed by analyzing light that is reflected back onto the proj ection element and coupled back into the waveguide structure .
Further possible applications for a display device according to the improved concept besides smart eyeglasses are fields , where a content is being displayed on a transparent screen while requiring sensing functionality . Further examples for such applications are : smart windows for dynamic displays , 3- D advertisement screens , TV screens with embedded sensing, heads up displays with integrated sensing functionality, and virtual cockpits in automotive with integrated sensing, e . g . for eye-tracking or fatigue detection purposes , for instance .
The aforementioned obj ect is further achieved by a method for operating a display device . The method comprises providing a display area on or within a transparent substrate of a display device , wherein the display area comprises a bidirectional proj ection element . The method further comprises providing, by means of a processor, an image signal to a transmitter unit , generating, by means of the transmitter unit , a light signal based on the image signal , and providing by means of an optical waveguide structure , the light signal to the bidirectional proj ection element .
The method according to the improved concept further comprises directing, by means of the bidirectional proj ection element , the light signal to an eye of a user, and receiving light that is reflected from the eye . The method further comprises providing, by means of the optical waveguide structure , the received light to a receive unit , and generating, by means of the receiver unit , a detection signal based on the received light . The method further comprises processing, by means of the processor, the detection signal . Further embodiments of the method become apparent to the skilled reader from the embodiments of the display device as described above .
The following description of figures of exemplary embodiments may further illustrate and explain aspects of the improved concept . Components and parts of the display device with the same structure and the same ef fect , respectively, appear with equivalent reference symbols . Insofar as components and parts of the display device correspond to one another in terms of their function in di f ferent figures , the description thereof is not repeated for each of the following figures .
In the figures :
Figure 1 shows a schematic view of an exemplary embodiment of a display device according to the improved concept ;
Figures 2 and 3 show further exemplary embodiments of a display device according to the improved concept ;
Figures 4 and 5 illustrate di f ferent configurations of the transmitter and receiver units and of the waveguide structure ; and
Figures 6 to 10b illustrate various embodiments of waveguide coupling employed in a display device according to the improved concept .
Figure 1 shows a schematic view of an exemplary embodiment of a display device 1 according to the improved concept . The display device 1 comprises a transparent substrate 10 , which is a glass or a polymer substrate , for instance . Transparent in this context means that the substrate is transmissive across the visible wavelength range of electromagnetic radiation . Typical material choices for the transparent substrate in the domain of wearable devices are optical crown glass , such borosilicate or BK7 , and allyl diglycol carbonate , also known as ADC or CR-39 . Particularly in automotive applications , the transparent substrate can be a window or more speci fically a windshield made of laminated safety glass .
On a surface of the substrate 10 , a waveguide structure 13 is arranged comprising optical waveguides 13a, having waveguide cores , and a cladding 13b . The waveguide structure 13 is configured to guide electromagnetic waves in the optical spectrum, in particular in the visible domain and optionally in a subrange of the infrared domain, e . g . the near infrared domain, along a main direction of extension of the waveguide structure 13 . Optical waveguides themselves are a well-known concept and are not further detailed throughout this disclosure .
In this embodiment , the bidirectional proj ection element 12 is arranged within the cladding 13b and is coupled, in particular directly coupled, to the optical waveguides 13a . The bidirectional proj ection element 12 is configured to direct light received from the optical waveguides 13a to an eye 2 of a user of the display device 1 . In other words , the bidirectional proj ection element 12 defines a display area 11 , in which visual content , e . g . virtual augmented reality content , is formed from the perspective of the user . For example , the bidirectional proj ection element 12 is a di f fractive optical element , in which each row or column is addressed by one optical waveguide 13a of the waveguide structure 13. Alternatively, the bidirectional projection element 12 is a holographic optical element. As illustrated, the bidirectional projection element 12 can be waveguide based. For example, the bidirectional projection element 12 constitutes an extension of the optical waveguides 13, wherein a core region is patterned or structured to form the bidirectional projection element 12.
As illustrated with the two arrows, the bidirectional projection element 12 is further configured to receive light from the eye 2 of the user and couple this light to the optical waveguides 13a of the optical waveguide structure 13. In this sense, the bidirectional projection element 12 can be understood as a bidirectional optical coupling element such as a Bragg grating coupling element. For example, the display area 11 is formed by a waveguide display comprising optical waveguides 13a and a bidirectional projection element 12. In alternative embodiments, the optical waveguide structure 13 and the projection element 12 can be arranged within a transparent substrate 10.
For example, the light received by the projection element 12 is light that is emitted by the transmitter unit 14a, guided via the waveguide structure 13 and the projection element 12 to the user's eye 2, and reflected back, e.g. from the retina or the iris, towards the projection element 12. In particular, the transmitter unit 14a can be configured to emit visible light for forming an image, e.g. a virtual content image, on the display surface 11, and to emit infrared light, in particular near-infrared light, that is directed to the user's eye 2, reflected back to the bidirectional projection element 12 and guided to the receiver unit 14b for detection . This signal can be processed by means of the processor 15 , for example , for determining a direction of view, an accommodation of the eye 2 , an identity of the user via iris recognition, and/or a cardiac cycle of the user . Therein, the transmitter unit 14a can be configured to emit pulses of infrared light , e . g . with a period of around 60 ms constituting the typical reaction time of a human eye .
In order to enable the view of an ambient environment 3 of the display device 1 , the display area 11 comprising the proj ection element 12 is at least partially transparent with respect to the visible domain, e . g . a wavelength between 400 nm and 800 nm . Hence , in this embodiment likewise the optical waveguides 13a, the cladding 13b and the proj ection element 12 are at least partially transparent .
The display device 1 further comprises a transceiver unit 14 comprising a transmitter unit 14a and a receiver unit 14b . Therein, the transceiver unit 14 is coupled to , in particular directly coupled to , the waveguide structure 13 . For example , the transceiver unit 14 comprises a silicon chip or substrate having an integrated circuit for controlling the transmitter unit 14a and the receiver unit 14b . Circuitry elements of the transceiver unit 14 can be coupled to the transmitter unit 14a and the receiver unit 14b via interconnection elements , such as wire bonds or TSVs in combination with solder bumps , for instance .
The transmitter unit 14a for example comprises light emitters , such as micro-LEDs or micro VCSELs , configured to illuminate pixels of the display area 11 . In other words , the transmitter unit 14a comprises an array, e . g . a one- dimensional array, of light emitters that are coupled to the bidirectional proj ection element 12 via optical waveguides 13a of the waveguide structure 13 . The transmitter unit 14a can be regarded a converter unit for converting a digital image signal into a visible light signal that is directed to the eye 2 of the user .
Analogously, the receiver unit 14b for example comprises light detectors , such as micro photodiodes , configured to detect light that is received by the bidirectional proj ection element 12 and guided to the receiver unit 14b via the optical waveguide structure 13 . In other words , the transmitter unit 14a comprises an array, e . g . a onedimensional array, of light detectors that are coupled to the bidirectional proj ection element 12 via optical waveguides 13a of the waveguide structure 13 . The receiver unit 14b can be regarded a converter unit for converting a visible light signal received from the eye 2 of the user into a digital detection signal .
The receiver unit 14b can be configured to detect light captured by the bidirectional proj ection element 12 within a predefined subrange of the display area 11 . For example , the receiver unit 14b receives light captured by the bidirectional proj ection element 12 within a 200 to 200 pixels subrange .
The display device 1 further comprises a processor 15 that is coupled to the transceiver unit 14 , and is configured to control the transmitter unit 14a via providing the aforementioned digital image signal representing an image to be displayed on the display area 11 . The processor 15 is further configured to receive from the receiver unit 14b the digital detection signal and process the signal according to a predefined process . For example , the processor 15 is configured to apply an eye-tracking or iris authentication process on the digital detection signal .
Figures 2 and 3 show a further exemplary embodiments of a display device 1 , which in these cases are configured as a wearable display device 1 , such as smart eyeglasses . The display device 1 comprises a spectacle frame 16a supporting the eyeglass lenses 16b, which each act as a transparent substrate 10 according to the improved concept . Within or on the eyeglass lenses 16b, the bidirectional proj ection element 12 defining the display area 11 is arranged . As described in the previous figure , the bidirectional proj ection element 12 is coupled to the transmitter unit 14a and to the receiver unit 14b via the waveguide structure 13 .
In the embodiment of Figure 2 , the transmitter unit 14a and the receiver unit 14b are designed as separate units . As the spectacle frame 16a can comprise cavities for providing housing for the transmitter and receiver units 14a, 14b, having separate units can be desirable in order to arrange both components within the spectacle frame 16a in an inconspicuous manner .
In the embodiment of Figure 3 , the aforementioned transceiver unit 14 includes the transmitter and receiver units 14a, 14b in a single unit . This in turn uses the concept of the optical waveguides 13a operating in a bidirectional manner . In other words , the optical waveguides 13a transmit light signals from the transmitter unit 14a to the bidirectional proj ection element 12 and from the bidirectional proj ection element 12 to the receiver unit 14b . In comparison to the embodiment of figure 2 , ef fectively only hal f the amount of optical waveguides 13a of the waveguide structure 13 is required .
Figure 4 illustrates a first configuration of the transmitter and receiver units 14a, 14b and of the waveguide structure 13 of a display device 1 according to the improved concept . In this first embodiment , a transceiver 14 comprising a transceiver integrated circuit , e . g . arranged on a silicon chip or substrate , comprises an integrated receiver unit 14b that is coupled to a receiving portion of the waveguide structure 13 .
The transmitter unit 14a, on the other hand, comprises a dedicated chip on an optical substrate , e . g . an A12O3 , GaAs , InP, which is electrically connected to a driver chip or IC, which preferably is made from silicon . The transmitter unit 14a is coupled to the transceiver 14 via interconnects such as wire bonds or TSVs and bumps . Moreover, the transmitter unit 14a is coupled to a transmitting portion of the waveguide structure 13 .
The transmitter unit 14a comprises an array of light emitters , such as a MEMS array of lasers . Likewise , the receiver unit 14b comprises an array of lights detectors , such as mems array of micro photodiodes .
This first configuration shown in figure 4 corresponds to that illustrated in the embodiment of the display device 1 of figure 2 , for example , in which the waveguide structure 13 is operated in a unidirectional manner . The second configuration of the transmitter and receiver units 14a, 14b and of the waveguide structure 13 of a display device 1 illustrated in figure 5 , in contrast corresponds to that illustrated in the embodiment of the display device 1 of figure 3 , for example .
In this configuration, the waveguide structure 13 is operated in a bidirectional manner . For example , both the transmitter unit 14a and the receiver unit 14b are arranged on a common substrate , which may be an A12O3 , GaAs , InP, which is electrically connected to a driver chip or IC, which preferably is made from silicon . The transmitter unit 14a and the receiver unit 14b, analogous to figure 4 , are coupled to the transceiver 14 comprising a transceiver integrated circuit via interconnects such as wire bonds or TSVs and bumps .
It is pointed out that the capturing regions illustrated by the arrows in figures 4 and 5 serve illustration purposes only . Actual optical coupling regions preferably directly coupled to the waveguide structure 13 to the transmitter unit 14a and the receiver unit 14b, particularly without any free- space path or optical elements such as mirrors or prisms arranged in between . Actual optical coupling regions according to the improved concept are discussed in the following figures 6 to 10b .
Figure 6 illustrates a waveguide splitter or combiner particularly used in embodiments of the display device 1 , in which the waveguide structure 13 is operated in a bidirectional manner . This way, cross talk between the light that this to be guided from the transmitter unit 14a to the bidirectional proj ection element 12 and light that is to be guided from the bidirectional proj ection element 12 to the receiver unit 14b is minimi zed . For further minimi zation of said cross talk, alternatively or in addition to an optical circulator, in particular a waveguide-based optical circulator, can be employed . The propagation direction of light as illustrated by means of the arrows in figure 6 .
Figure 7 schematically illustrates a first waveguide coupling configuration employed in a display device 1 according to the improved concept . Speci fically, figure 7 illustrates vertical coupling of the waveguide structure 13 to the transmitter unit 14a . For simpli fication, the transmitter unit 14a is shown to comprise a single emission layer 142 buried within a cladding 141 and arranged on a substrate 140 , while the waveguide structure comprises a single waveguide having a waveguide core 13a inside the cladding 13b coupled to the emission layer 142 . For example , the emission layer 142 reali zes a VCSEL or a micro-LED . In particular, the optical waveguide core 13a is directly vertically coupled to the transmitter engine 14a .
Figure 8 , on the other hand, schematically illustrates a second waveguide coupling configuration, in which the optical waveguide core 13a is directly coupled to the transmitter engine 14a via lateral butt-coupling . In this embodiment , the transmitter unit 14a is shown to comprise a single emission layer 142 that reali zes a laterally emitting light source , e . g . an LED, a DFB laser, or a DBR laser .
Figure 9 schematically illustrates the first waveguide coupling configuration for the receiver unit 14b . Analogous to the configuration of figures 7 , for illustration purposes the receiver unit 14b is shown to comprise a single detection layer 143 within a substrate 140 , e . g . a silicon substrate , and covered by an oxide cladding 141 , for instance . Also the receiver unit 14 B is directly coupled to the waveguide structure 13 via vertical coupling in this configuration .
Figure 10a schematically illustrates a cross-sectional view of a second waveguide coupling configuration for the receiver unit 14b . In this lateral coupling configuration, the receiver unit 14b is arranged with respect to the waveguide structure 13 in a lateral manner . The dashed line illustrates the light path . Optionally, the waveguide cladding 13 B can be at least partially removed in the region of the receiver unit 14b to enhance the detection ef ficiency . Like in the previous figure , the detection layer 143 is arranged within a substrate 140 and covered by a silicon dioxide backend of line cladding 141 .
Figure 10b schematically illustrates a top view of the second waveguide coupling configuration of figure 10b . To further enhance the detection ef ficiency of the detection layer 143 , the waveguide core 13a can have a tapered profile in the detection region, i . e . in the region of the receiver unit 14b .
It is noted that the coupling configuration illustrated in figures 6 to 10b merely serve illustration purposes . Actual embodiments of the transmitter unit 14a and the receiver unit 14b can comprise a plurality of light emitters and light detectors . For example , the transmitter unit 14a and the receiver unit 14b comprise arrays of light emitters and light detectors , respectively . Likewise , the waveguide structure 13 coupled to the transmitter unit 14a and to the receiver unit 14b comprises a plurality of optical waveguides , for example matching an amount of the plurality of light emitters and detectors .
The embodiments of the display device 1 and embodiments of its operation method shown in the figures represent exemplary embodiments , therefore they do not constitute a complete list of all embodiments according to the improved concept . Actual display devices may vary from the embodiments shown in terms of additional components , shape and configuration, for instance . In particular, features shown in the various figures may be combined with each other and hence form additional embodiments according to the improved concept .
This patent application claims priority to German patent application 10 2020 132 325 . 5 , the disclosure content of which is hereby incorporated by reference .
Reference symbols
1 display device
2 eye
3 ambient environment
10 transparent substrate
11 display area
12 bidirectional proj ection element
13 waveguide structure
13a optical waveguide
13b waveguide cladding
14 transceiver unit
14a transmitter unit
14b receiver unit
15 processor
16a spectacle frame
16b eyeglass lens
140 substrate
141 cladding
142 emission layer
143 detection layer

Claims

- 24 - Claims
1. A display device (1) , comprising: a transparent substrate (10) ; a display area (11) on or within the substrate (10) comprising a bidirectional projection element (12) configured to direct a light signal to an eye (2) of a user and to receive light that is reflected from the eye (2) , wherein the display area (11) is at least partially transparent and is configured to enable viewing of an ambient environment (3) through the display area (11) ; a transmitter unit (14a) configured to generate the light signal based on an image signal; a receiver unit (14b) configured to generate a detection signal based on the light received by the bidirectional projection element (12) ; an optical waveguide structure (13) coupling the transmitter unit (14a) and the receiver unit (14b) to the bidirectional projection element (12) ; and a processor (15) configured to provide the image signal to the transmitter unit (14a) and to process the detection signal from the receiver unit (14b) .
2. The display device (1) according to claim 1, wherein the transmitter unit (14a) and the receiver unit (14b) are comprised by a transceiver unit (14) and the optical waveguide structure (13) is a bidirectional optical waveguide structure coupling the transceiver unit (14) to the bidirectional projection element (12) .
3. The display device (1) according to claim 1 or 2, wherein the transmitter unit (14a) comprises an array, in particular a one-dimensional array, of light sources.
4. The display device (1) according to one of claims 1 to 3, wherein the receiver unit (14b) comprises an array, in particular a one-dimensional array, of light detectors.
5. The display device (1) according to one of claims 1 to 4, wherein the bidirectional projection element (12) comprises a diffractive grating or a holographic optical element.
6. The display device (1) according to one of claims 1 to 5, wherein the transmitter unit (14a) is configured to emit light in the visible domain and in the infrared, in particular the near-infrared, domain.
7. The display device (1) according to one of claims 1 to 6, wherein the light directed to the eye (2) comprises visual content in the visible domain and an infrared, in particular a near-infrared, signal.
8. The display device (1) according to one of claims 1 to 7, wherein the receiver unit (14b) is configured to generate the detection signal based on infrared, in particular nearinfrared, light.
9. The display device (1) according to one of claims 1 to 8, wherein the transmitter unit (14a) and the receiver unit
(14b) are coupled to the optical waveguide structure (13) via vertical coupling or lateral butt-coupling.
10. The display device (1) according to one of claims 1 to 9, wherein the processor (15) is further configured to determine from the detection signal at least one of: a direction of view and/or an accommodation of the eye (2) ; an identity of the user via iris recognition of the eye
( 2 ) ; and a cardiac cycle of the user.
11. The display device (1) according to one of claims 1 to
10, wherein the receiver unit (14b) is configured to generate the detection signal based on light that is received by the bidirectional projection element (12) within a partial area of the display area (11) .
12. The display device (1) according to one of claims 1 to 11, further comprising a spectacle frame (16a) , wherein the transparent substrate (10) is an eyeglass lens (16b) supported by the spectacle frame (16a) .
13. The display device (1) according to claim 12, wherein the spectacle frame (16a) acts as housing for the transmitter unit (14a) , the receiver unit (14b) and the processor (15) .
14. The display device (1) according to one of claims 1 to 13, wherein an optical path between the eye (2) and the bidirectional projection element (12) is the only free space optical path.
15. The display device (1) according to one of claims 1 to 14, wherein the display device (1) is free of a dedicated detection system for monitoring a direction of view of the eye .
16. The display device (1) according to one of claims 1 to
15, wherein the optical waveguide structure (13) comprises a plurality of optical waveguides (13a) . 27
17. The display device (1) according to claim 16, wherein the plurality of optical waveguides (13a) are arranged parallel to each other.
18. A method for operating a display device (1) , the method comprising : providing a display area (11) on or within a transparent substrate (10) of the display device (1) , the display area (11) having a bidirectional projection element (12) ; providing, by means of a processor (15) , an image signal to a transmitter unit (14a) ; generating, by means of the transmitter unit (14a) , a light signal based on the image signal; providing, by means of an optical waveguide structure (13) , the light signal to the bidirectional projection element ( 12 ) ; directing, by means of the bidirectional projection element (12) , the light signal to an eye (2) of a user; receiving, by means of the bidirectional projection element (12) , light that is reflected from the eye (2) ; providing, by means of the optical waveguide structure (13) , the received light to a receiver unit (14b) ; and generating, by means of the receiver unit (14b) , a detection signal based on the received light; and processing, by means of the processor (15) , the detection signal .
PCT/EP2021/082123 2020-12-04 2021-11-18 Display device and method for operating a display device WO2022117355A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102020132325 2020-12-04
DE102020132325.5 2020-12-04

Publications (1)

Publication Number Publication Date
WO2022117355A1 true WO2022117355A1 (en) 2022-06-09

Family

ID=78819516

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2021/082123 WO2022117355A1 (en) 2020-12-04 2021-11-18 Display device and method for operating a display device

Country Status (1)

Country Link
WO (1) WO2022117355A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130077049A1 (en) * 2011-09-26 2013-03-28 David D. Bohn Integrated eye tracking and display system
US20190041634A1 (en) * 2016-02-04 2019-02-07 Digilens, Inc. Holographic Waveguide Optical Tracker
US20190056600A1 (en) * 2016-12-31 2019-02-21 Lumus Ltd Eye tracker based on retinal imaging via light-guide optical element
US20190179409A1 (en) * 2017-12-03 2019-06-13 Frank Jones Enhancing the performance of near-to-eye vision systems

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130077049A1 (en) * 2011-09-26 2013-03-28 David D. Bohn Integrated eye tracking and display system
US20190041634A1 (en) * 2016-02-04 2019-02-07 Digilens, Inc. Holographic Waveguide Optical Tracker
US20190056600A1 (en) * 2016-12-31 2019-02-21 Lumus Ltd Eye tracker based on retinal imaging via light-guide optical element
US20190179409A1 (en) * 2017-12-03 2019-06-13 Frank Jones Enhancing the performance of near-to-eye vision systems

Similar Documents

Publication Publication Date Title
EP3660575B1 (en) Eye tracking system and eye tracking method
EP3602178B1 (en) Method and system for tracking eye movement in conjunction with a light scanning projector
US9285877B2 (en) Heads-up display
EP3575851B1 (en) Information display device and information display method
CN107533226B (en) Near-to-eye display with stacked light guides
US20200033590A1 (en) Image generating device and image generating method
US20220382064A1 (en) Metalens for use in an eye-tracking system of a mixed-reality display device
WO2013049012A1 (en) Integrated eye tracking and display system
KR20150013815A (en) Head-mounted display with an eyeball-tracker integrated system
JP2006344915A (en) Optical unit
EP3690516A1 (en) Lens for eye-tracking and device with such a lens
EP3690500A1 (en) Lens for eye-tracking applications and head-worn device
WO2022117355A1 (en) Display device and method for operating a display device
US11426070B2 (en) Infrared illuminator and related eye tracking apparatus and method
KR20220046494A (en) Eye tracking method and eye tracking sensor
US7099536B1 (en) Single lens system integrating both transmissive and reflective surfaces for light focusing to an optical fiber and light reflection back to a monitor photodetector
US20230064721A1 (en) Scanning self-mixing interferometry with waveguide
US20140211320A1 (en) Visual line detection device
CN105242773B (en) A kind of electronic equipment
EP3690501A1 (en) Lens for eye-tracking applications and head-worn device
US7092597B2 (en) Bidirectional transmitting and receiving device
US11796829B1 (en) In-field illuminator for eye depth sensing
JP2023138124A (en) Inclination detector, visual line detector, head mount display, retina projection display device, optometer, user state estimation device and drive support system
JP2005116867A (en) Optical signal transmitter
CN113589940A (en) Human eye tracking system and virtual reality display device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21816000

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21816000

Country of ref document: EP

Kind code of ref document: A1