KR20230165389A - Method of Calibration for Holographic Energy Directing Systems - Google Patents

Abstract

The holographic energy directing system may include waveguide arrays and relay elements. The disclosed calibration method enables mapping of energy positions and mapping of energy positions to a defined angular direction of energy in a four-dimensional plenoptic system. Distortions due to waveguide arrays and relay elements can also be compensated.

Description

Calibration method for holographic energy directing systems {Method of Calibration for Holographic Energy Directing Systems}

The present invention relates generally to methods for calibrating holographic energy systems, and more particularly to methods for calibrating relay elements and waveguide elements of holographic energy systems.

Popularized by Gene Roddenberry's Star Trek and first conceived by author Alexander Moszkowski in the early 1900s, the dream of interactive virtual worlds within "holodeck" chambers has been a science topic for nearly a century. Inspired novels and technological innovations. However, there is no compelling embodiment of this experience outside of literature, media, and the collective imagination of children and adults.

One embodiment of the invention relates to a calibration method for an energy relay element, wherein the energy relay element is configured such that energy propagating through the energy relay element has a higher transmission efficiency in longitudinal orientation. The method includes receiving data of energy properties of energy at a first plurality of energy positions on a first surface of an energy relay element, wherein the energy at the first plurality of energy positions is distributed along a longitudinal orientation of the energy relay element. and receiving, wherein the energy is relayed from a second plurality of locations. The method further includes correlating predetermined data of energy properties of the energy in the second plurality of energy locations with data of energy properties of the energy in the first plurality of energy locations to generate a calibrated relay function. In one embodiment, the calibrated relay function includes a mapping of energy properties at a first plurality of energy locations to energy properties at a second plurality of energy locations.

One embodiment of a calibration method for an energy waveguide array is disclosed, wherein the energy waveguide array is operable to direct energy along uninhibited energy propagation paths extending from a first side to a second side of the energy waveguide array, , the uninhibited energy propagation paths extend to a plurality of energy positions on the first side and along a different angular direction with respect to the energy waveguide array depending on each energy position on the first side, on the second side. The method may include receiving data of energy properties of energy along uninhibited energy propagation paths on a second side of the waveguide array. The method correlates data of the energy properties of the energy at a plurality of energy locations with data of the energy properties of the energy along uninhibited energy propagation paths on the second side of the waveguide array, thereby producing a calibrated 4 for the energy waveguide array. A step of generating a dimensional (4D) plenoptic function may be further included. In one embodiment, the calibrated 4D plenoptic function includes a mapping between a plurality of energy locations and respective angular directions of uninhibited energy propagation paths.

A calibration method for an energy-directed system is disclosed, wherein an energy relay element of the energy-directed system is configured such that energy propagating through the energy relay element has a higher transmission efficiency in a longitudinal orientation, and an energy waveguide array of the energy-directed system is configured to transmit the energy. operable to direct energy along uninhibited energy propagation paths extending from a first side to a second side of the waveguide array, wherein the uninhibited energy propagation paths extend to a plurality of relayed energy locations on the first side; , which extends on the second side along different angular directions relative to the energy waveguide array depending on each energy location on the first side. The method includes receiving data of energy properties of energy at a plurality of relayed energy locations on a first surface of an energy relay element, wherein the energy at the first plurality of energy locations is transmitted through the energy relay element along a longitudinal orientation. and the step of receiving, relayed from a source energy location. The method comprises correlating predetermined data of energy properties of energy at a plurality of source energy locations with data of energy properties of energy at a plurality of relayed energy locations to generate a calibrated relay function, the calibrated relay function The step may further include generating the calibrated relay function comprising mapping energy properties at a first plurality of energy locations to energy properties at a second plurality of energies. The method includes receiving data of energy properties of energy along uninhibited energy propagation paths on a second side of the waveguide array and data of energy properties of the energy at a plurality of relayed energy locations, the second side of the waveguide array Generating a calibrated four-dimensional (4D) plenoptic function for the energy waveguide array by correlating data of energy properties of energy along non-inhibited energy propagation paths in a plane, wherein the calibrated 4D plenoptic function is a plurality of energy positions. The method may further include generating the calibrated 4D plenoptic function, including a mapping between the respective angular directions of the non-inhibited energy propagation paths.

It is included in the content of the present invention.

Embodiments are shown by way of example in the accompanying drawings, where like reference numerals indicate like parts:
1 is a schematic diagram showing design parameters for an energy directed system.
Figure 2 is a schematic diagram showing an energy system with an active device area with a mechanical envelope.
Figure 3 is a schematic diagram showing an energy relay system.
Figure 4 is a schematic diagram showing one embodiment of energy relay elements glued and secured together to a base structure.
Figure 5A is a schematic diagram showing an example of an image relayed over multi-core optical fibers.
Figure 5b is a schematic diagram showing an example of an image relayed through an energy relay demonstrating characteristics of the Transverse Anderson Localization principle.
Figure 6 is a schematic diagram showing rays propagating from the energy surface to the viewer.
Figure 7 is a schematic diagram showing a system architecture operable to direct energy according to four-dimensional plenoptic functions.
8 is a flow diagram illustrating a process for mapping energy locations and energy propagation paths for a four-dimensional plenoptic energy directing system.
Figure 9 is a schematic diagram showing a calibration system for calibrating energy relay elements in a four-dimensional plenoptic energy directing system.
Figures 10A-10C are embodiments of mappings in the process of Figure 8.
Figure 11 is a flow diagram illustrating one embodiment of a process for mapping energy locations.
Figure 12 is a flow diagram illustrating another embodiment of a process for mapping energy locations.
FIG. 13 is a flow diagram illustrating one embodiment of a process for mapping energy locations and energy paths for a four-dimensional plenoptic energy directing system.
Figure 14 is a schematic diagram showing a calibration system for calibrating energy waveguide elements in a four-dimensional plenoptic energy directing system.

Although the making and use of various embodiments of the present disclosure are discussed in detail below, it should be understood that the present invention provides many applicable inventive concepts that may be implemented in a variety of specific situations. The specific embodiments discussed herein are merely illustrative of specific ways to make and use the present disclosure and do not limit the scope of the invention.

Embodiments of the holodeck (collectively referred to as “Holodeck Design Parameters”) provide sufficient energy stimulation to fool human sensory receptors into believing that the energy stimulation received within the virtual social interactive environment is real; , provides: 1) binocular disparity without external accessories, head-mounted eyewear or other peripheral devices; 2) accurate motion parallax, occlusion and opacity across the viewing volume for an arbitrary number of observers simultaneously; 3) visual focus through synchronous convergence, accommodation, and miosis of the eyes for all perceived rays; and 4) converging energy wave propagation of sufficient density and resolution that exceeds the human sensory “resolution” for sight, hearing, touch, taste, smell and/or balance.

It can serve all receptive fields in a powerful way as suggested by the Holodeck design parameters, including visual, auditory, somatosensory, gustatory, olfactory and vestibular systems. The technology that exists, based on prior art to date, is decades, if not centuries, old.

In this disclosure, the terms light field and hologram may be used interchangeably to define energy propagation upon stimulation of any sensory receptor response. Initial disclosures may refer to examples of electromagnetic and mechanical energy propagation through energy surfaces for holographic imaging and volumetric haptics, while all types of sensory receptors are envisioned in this disclosure. Additionally, the principles disclosed herein for energy propagation along propagation paths may be applicable to both energy emission and energy capture.

Generalized as lenticular printing, Pepper's Ghost, glasses-free stereoscopic displays, horizontal parallax displays, head-mounted VR and AR displays (HMDs), and "fauxlography". Many technologies exist today that are often unfortunately confused with holograms, including these other illusions. Although these technologies may exhibit some of the desired characteristics of a true holographic display, the ability to stimulate human visual sensory responses in a random manner sufficient to address at least two of the four identified holodeck design parameters. This is not enough.

These challenges have not been successfully implemented by prior art to create a seamless energy surface sufficient for holographic energy propagation. Includes parallax barriers, hogels, voxels, diffractive optics, multi-view projection, holographic diffusers, rotating mirrors, multilayer displays, time-sequential displays, head-mounted displays, etc. Although there are various approaches to implement volumetric and direction multiplexed light field displays, conventional approaches may involve compromises on image quality, resolution, angular sampling density, size, cost, safety, frame rate, etc. Ultimately, this may result in an unfeasible technology.

To achieve the holodeck design parameters for the visual, auditory, and somatosensory systems, the human acuity of each system is studied and understood to propagate energy waves sufficiently to fool human sensory receptors. . The visual system can resolve to about 1 arc minute, the auditory system can distinguish differences in placement by as little as 3 degrees, and the somatosensory system in the hand can distinguish points separated by 2-12 mm. You can. Although there are various and conflicting ways to measure this sensitivity, these values are sufficient to understand systems and methods for stimulating the perception of energy propagation.

Among the sensory receptors mentioned, the human visual system is by far the most sensitive, considering that even single photons can trigger sensations. For this reason, most of this introduction will focus on visual energy wave propagation, and fairly low resolution energy systems coupled within the disclosed energy waveguide surfaces can converge appropriate signals to induce holographic sensory perception. Unless otherwise stated, all disclosures apply to all energy and sensory domains.

When calculating the effective design parameters of energy propagation for a visual system given the viewing volume and viewing distance, the desired energy surface can be designed to contain an effective energy location density of many gigapixels. For large viewing volumes or near field viewing, design parameters of the desired energy surface may include an effective energy location density of hundreds of gigapixels or more. In comparison, the desired energy source is designed to have an energy site density of 1 to 250 megapixels for ultrasonic propagation of volume haptics or an array of 36 to 3,600 effective energy sites for acoustic propagation of holographic sounds depending on input environmental variables. It can be. An important point to note is that using the disclosed bidirectional energy surface architecture, all components can be configured to form suitable structures for arbitrary energy regions to enable holographic propagation.

However, major challenges for using holodecks today involve limitations in the available visual technologies and electromagnetic devices. Hearing and ultrasound devices, the complexity of which should not be underestimated, are less difficult given the order of magnitude difference in the desired density based on the sensory acuity in each receptive field. Holographic emulsions exist at resolutions exceeding the desired densities for encoding interference patterns in static images, but state-of-the-art display devices are limited by resolution, data throughput, and manufacturability. To date, native display devices have not been able to meaningfully generate light fields with nearly holographic resolution for vision.

Fabricating a single silicon-based device capable of meeting the desired resolution for a powerful light field display may not be practical and may involve extremely complex manufacturing processes beyond current manufacturing capabilities. Limitations to tiling multiple existing display devices together are shaped by the physical dimensions of packaging, electronics, enclosures, optics, and a host of other challenges that inevitably render the technology unfeasible from an imaging, cost, and/or size perspective. It involves seams and gaps.

Embodiments disclosed herein can provide a practical path for building a holodeck.

Exemplary embodiments will now be described with reference to the accompanying drawings, which form a part of this specification and illustrate example embodiments in which they may be practiced. As used in the disclosure and appended claims, the terms “embodiment,” “exemplary embodiment,” and “exemplary embodiment” do not necessarily refer to a single embodiment, but rather a range or range of exemplary embodiments. Without departing from the spirit, various exemplary embodiments can be easily combined and interchanged. Additionally, the terminology used herein is for the purpose of describing example embodiments only and is not intended to be limiting. In this regard, as used herein, the term "in" may include "in" and "on" and the terms "a", "an" and "the" may include singular and plural references. You can. Additionally, as used herein, the term “by” may also mean “from” depending on the context. Additionally, as used herein, the term “if” may also mean “if” or “when” depending on the context. Additionally, as used herein, the word “and/or” can refer to and encompass any and all possible combinations of one or more of the associated listed items.

Holographic system considerations:

* Overview of light field energy propagation resolution

Light fields and holographic displays are the result of multiple projections whose energy surface positions provide angular, color and intensity information propagated within a viewing volume. The disclosed energy surface provides opportunities for additional information to coexist and propagate through the same surface, eliciting different sensory responses. Unlike stereoscopic displays, the visible position of the converged energy propagation paths in space does not change as the viewer moves around the viewing volume, and an arbitrary number of viewers can view the propagated objects in real space as if they were actually there. can be viewed at the same time. In some embodiments, the propagation of energy may be located in the same energy propagation path but in opposite directions. For example, both energy release and energy capture along the energy propagation path are possible in some embodiments of the invention.

1 is a schematic diagram showing variables related to stimulation of sensory receptor responses. These variables include surface diagonal (101), surface width (102), surface height (103), determined target seating distance (118), and target seating field (104) of the view field from the center of the display. ), the number of intermediate samples shown here as samples between the eyes (105), the average adult interocular distance (106), the average resolution of the human eye per minute (107), the horizontal field of view formed between the target observer position and the surface width. (108), the vertical field of view formed between the target observer position and the surface height (109), the total number of resulting horizontal waveguide element resolutions or elements across the surface (110), the resulting vertical waveguide element resolution or elements across the surface. total number of 111, sample distance 112 based on the inter-ocular spacing and the number of intermediate samples for the angle projection between the eyes, angular sampling 113 which may be based on the sample distance and target seating distance, desired A device wherein the device is a full resolution horizontal degree per waveguide element derived from angular sampling (114), a full resolution vertical degree per waveguide element derived from desired angular sampling, 115, and a count of a determined number of desired discrete energy sources. It may include a horizontality 116, and a device verticality 117 which is a coefficient of a determined number of desired discrete energy sources.

One way to understand the minimum desired resolution is to use the following criteria to ensure sufficient stimulation of visual (or other) sensory receptor responses: surface size (e.g., 84" diagonal), surface aspect ratio (e.g., 16:9), Seating distance (e.g., 128" from the display), seating field of view (e.g., 120 degrees or +/-60 degrees centered on the center of the display), desired intermediate samples at a distance (e.g., one addition between the eyes) potential transmission path), the average interocular distance of an adult (approximately 65 mm), and the average resolution of the human eye (approximately 1 arcminute). These example values should be considered placeholders according to specific application design parameters.

Additionally, each of the values attributed to the visual sensory receptors can be replaced by other systems to determine the desired propagation path parameters. For other embodiments of energy propagation, it can be considered that the angular sensitivity of the auditory system may be as low as 3 degrees, and the spatial resolution of the somatosensory system of the hand may be as small as 2-12 mm.

Although there are various and conflicting ways to measure this sensory acuity, these values are sufficient to understand systems and methods for stimulating the perception of virtual energy propagation. There are many ways to consider design resolution, and the method proposed below combines practical product considerations with the biological degradation limits of sensory systems. As can be appreciated by those skilled in the art, the following outline is a simplification of the design of any such system and should be considered for illustrative purposes only.

With the resolution limits of the sensory system understood, the total energy waveguide element density can be calculated such that the receiving sensory system cannot distinguish a single energy waveguide element from adjacent elements, as follows:

Surface aspect ratio =

The above calculations result in a field of view of approximately 32Х18°, which results in approximately 1920Х1080 (rounded to the nearest fraction) energy waveguide elements being required. Additionally, both (u, v) variables can be constrained such that the field of view is constant to provide a more regular spatial sampling of energy positions (e.g., pixel aspect ratio). The angular sampling of the system assumes a target visual volume location and additional propagation energy paths defined between two points at an optimized distance, as follows:

In this case, any metric can be utilized to describe the appropriate number of samples for a given distance, but the interocular distance is utilized to calculate the sample distance. Considering the above variables, approximately 1 ray per 0.57° may be required, and the resolution of the entire system per independent sensory system can be determined as follows:

Using the above scenario, taking into account the size of the energy surface and angular resolution resolved for the vision system, the resulting energy surface would preferably contain energy resolution positions of about 400kХ225k pixels, or a holographic propagation density of 90 gigapixels. You can. These parameters provided are for illustrative purposes only, and many other sensory and energy metrology considerations must be considered for optimization of holographic propagation of energy. In a further embodiment, energy resolution positions of 1 gigapixel may be desirable based on input variables. In a further embodiment, 1,000 gigapixel energy resolution positions may be required based on the input variables.

Current technology limitations:

Active area, device electronics, packaging and mechanical envelope

Figure 2 shows a device 200 having an active area 220 with a given mechanical form factor. Device 200 may include electronics 240 and a driver 230 for powering and interfacing with active region 220, with dimensions indicated by x and y arrows. The mechanical footprint of the device 200 can be further minimized by introducing flex cables into the device 200 without considering cabling and mechanical structures for driving, powering, and cooling the components. The minimal footprint of this device 200 may also be referred to as mechanical envelope 210 with dimensions indicated by the M:x and M:y arrows. This device 200 is for illustrative purposes only and custom electronic design may further reduce mechanical envelope overhead, but in almost all cases may not be the exact size of the active area of the device. In one embodiment, the device 200 represents the dependence of the electronics on the active image area 220 for a micro OLED, DLP chip or LCD panel, or any other technology with the purpose of image illumination.

In some embodiments, it may also be possible to aggregate multiple images into a larger overall display taking into account other projection techniques. However, this may come at the cost of greater complexity in terms of throw distance, minimum focus, optical quality, uniform field resolution, chromatic aberration, thermal characteristics, calibration, alignment, and additional size or form factor. there is. For most practical applications, hosting dozens or hundreds of these projection sources 200 may result in a much larger design with less reliability.

For illustrative purposes only, assuming energy devices with an energy site density of 3840Х2160 sites, one can determine the number of individual energy devices (e.g., devices 100) required for an energy surface, given by:

Taking the above resolution considerations into account, approximately 105Х105 devices similar to those shown in Figure 2 are required. It should be noted that many devices contain various pixel structures that may or may not be mapped to a regular grid. If additional sub-pixels or locations exist within each total pixel, these may be utilized to create additional resolution or angular density. Additional signal processing may be used to determine how to transform the light field into the correct (u, v) coordinates, depending on the specified location of the pixel structure(s), which may be an explicit characteristic of each device as it is known and calibrated. there is. Additionally, different energy domains may involve different handling of these ratios and device structures, and one skilled in the art will understand the direct essential relationship between each of the desired frequency domains. This will be explained and discussed in more detail in later disclosures.

The resulting calculations can be used to understand how many of these individual devices are desired to produce a full-resolution energy surface. In this case, about 105Х105 or about 11,080 devices may be needed to achieve the vision threshold. Challenges and novelties exist within the process of creating seamless energy surfaces from these available energy locations for sufficient sensory holographic propagation.

Overview of seamless energy surfaces:

Construction and design of arrays of energy relays

In some embodiments, approaches are disclosed to solve the challenge of generating high energy site density from an array of individual devices that do not have shims due to limitations in mechanical structures for the devices. In one embodiment, an energy propagating relay system can increase the effective size of the active element area to meet or exceed mechanical dimensions to form an array of relays and form a unique seamless energy surface.

Figure 3 shows one embodiment of such an energy relay system 300. As shown, relay system 300 may include device 310 mounted on mechanical envelope 320, with energy relay elements 330 propagating energy from device 310. Relay element 330 may be configured to provide the ability to alleviate any gaps 340 that may be created when multiple mechanical envelopes 320 of devices are placed within an array of multiple devices 310.

For example, if the active area 310 of the device is 20mmХ10mm and the mechanical envelope 320 is 40mmХ20mm, then the energy relay element 330 is approximately 20mmХ10mm on the reduced end (arrow A) and 40mmХ20mm on the expanded end (arrow B). An array of these elements 330 can be designed at a scale of 2:1 to create a tapered shape, seamlessly forming an array of these elements 330 together without altering or colliding with the mechanical envelope 320 of each device 310. ) Provides the ability to sort. Mechanically, relay elements 330 may be bonded or fused together for alignment and polishing while ensuring minimal seam gap 340 between devices 310 . In one such embodiment, it is possible to achieve a seam gap 340 that is less than the visual acuity limit of the eye.

Figure 4 shows an example of a base structure 400 with energy relay elements 410 formed together and rigidly secured to an additional mechanical structure 430. The mechanical structure of the seamless energy surface 420 couples multiple energy relay elements 410, 450 in series to the same base structure through bonding or other mechanical processes to mount the relay elements 410, 450. Provides ability. In some embodiments, each relay element 410 may be fused, bonded, glued, pressure-fitted, aligned, or otherwise attached together to form the resulting seamless energy surface 420. In some embodiments, device 480 may be mounted rearward of relay element 410 and may be passively or actively aligned to ensure that proper energy position alignment is maintained within determined tolerances. there is.

In one embodiment, the seamless energy surface includes one or more energy sites, one or more energy relay element stacks include a first and second side, each energy relay element stack includes one or more energy sites and a seamless display surface. arranged to form an inherently seamless display surface that directs energy along propagation paths extending therebetween, wherein the separation distance between the edges of any two adjacent second surfaces of the distal energy relay elements is equal to or greater than that of the unique seamless display surface. It is smaller than the minimum perceptible contour defined by the visual acuity of the human eye with better than 20/100 visual acuity at a distance greater than the width.

In one embodiment, each of the seamless energy surfaces includes one or more energy relay elements each having one or more structures forming first and second surfaces having a transverse and longitudinal orientation. The first relay surface has a different area than the second relay surface that results in positive or negative magnification, and consists of explicit surface contours for both the first and second surfaces that pass energy through the second relay surface. and substantially fills an angle of +/-10 degrees with respect to the normal of the surface contour across the second relay surface.

In one embodiment, multiple energy domains may be configured within a single energy relay or between multiple energy relays to direct one or more sensory holographic energy propagation paths including visual, auditory, tactile or other energy domains.

In one embodiment, the seamless energy surface is comprised of energy relays comprising two or more first faces to each second face to simultaneously receive and emit one or more energy domains to provide bidirectional energy propagation throughout the system. It is composed.

In one embodiment, the energy relays are provided as loose coherent elements.

Introduction to component engineered structures:

Initiated advances in transverse Anderson omnipresent energy relays

The properties of energy relays can be significantly optimized according to the principles disclosed herein for energy relay elements leading to transverse Anderson localization. Transverse Anderson localization is the propagation of light rays transmitted through a material that is transversely disordered but longitudinally constant.

This suggests that the effect of materials inducing Anderson localization may be less due to total internal reflection than to randomization between multiple scattering paths, where wave interference continues in the longitudinal orientation while continuing in the transverse orientation. ) means that the spread to can be completely limited.

An important additional advantage is the elimination of cladding of traditional multi-core optical fiber materials. The cladding functionally eliminates the scattering of energy between the fibers, but at the same time acts as a barrier to rays of energy, so that at least a core to clad ratio (e.g. a core to clad ratio of 70:30) reduces the transmission of received energy. It reduces transmission by up to 70%) and additionally forms strong pixelated patterning in the propagated energy.

FIG. 5A is an example of such a non-Anderson Localization energy relay 500, where images are relayed through a multi-core optical fiber that may exhibit pixilation and fiber noise due to the inherent properties of optical fiber. Shows a cross-sectional view. Using conventional multi-mode and multi-core optical fiber, the relayed images are transmitted from the individual cores, where cross-talk between the cores reduces the modulation transfer function and increases blurring. Due to the overall internal reflection properties of the array, it can be inherently pixelated. The resulting images produced using conventional multi-core optical fibers tend to have residual static noise fiber patterns similar to those shown in Figure 3.

FIG. 5B shows the same relayed image 550 via an energy relay comprising materials where the relayed pattern exhibits characteristics of transverse Anderson localization with a greater density of grain structures, compared to the fixed fiber pattern from FIG. 5A . Shows an example. In one embodiment, relays containing randomized microscopic component engineered structures induce transverse Anderson localization and utilize higher resolvable resolution propagation than commercially available multimode glass optical fibers to transmit light more efficiently. send to

There are significant advantages to the transverse Anderson unidirectional material properties in terms of both cost and weight, where glass materials of similar optical grade cost and weigh 10 to 100 times more than the cost and weight for the same materials produced in the examples. , wherein the disclosed systems and methods include randomized microcomponent engineered structures that present significant opportunities for both cost and quality improvements over other techniques known in the art.

In one embodiment, a relay element exhibiting transverse Anderson localization may include a plurality of at least two different component engineered structures in each of three orthogonal planes arranged in a dimensional grid, the plurality of structures being transversely located within the dimensional grid. Randomized distributions of the material wave propagation properties in the plane and form channels of similar values of the material wave propagation properties in the longitudinal plane in the dimensional grid, where localized energy waves propagating through the energy relay have a transverse orientation. It has higher transmission efficiency in longitudinal orientation compared to

In one embodiment, multiple energy domains may be configured within a single energy relay or between multiple transverse Anderson omnipresent energy relays to direct one or more sensory holographic energy propagation paths including visual, auditory, tactile or other energy domains. You can.

In one embodiment, the seamless energy surface has a transverse Anderson localization comprising two or more first faces to each second face to simultaneously receive and emit one or more energy domains to provide bidirectional energy propagation throughout the system. It consists of energy relays.

In one embodiment, transverse Anderson localized energy relays are configured as loose coherent or flexible energy relay elements.

Considerations for 4D Plenoptic functions:

Selective propagation of energy through holographic waveguide arrays

As mentioned above and throughout this specification, light field display systems generally include an energy source (eg, an illumination source) and a seamless energy surface comprised of sufficient energy site density as noted in the description above. Multiple relay elements can be used to relay energy from energy devices to a seamless energy surface. Once energy is transferred to the seamless energy surface with the required energy position density, the energy can propagate according to a 4D plenoptic function through the disclosed energy waveguide system. As understood by those skilled in the art, 4D plenoptic functions are well known in the art and will not be described further herein.

The energy waveguide system transmits energy through a plurality of energy positions along a seamless energy surface representing the spatial coordinates of a 4D plenoptic function having a structure configured to change the angular direction of passing energy waves, representing the angular component of the 4D plenoptic function. propagates selectively, but the propagated energy waves can converge in space according to a plurality of propagation paths directed by a 4D plenoptic function.

Reference is now made to Figure 6, which shows an example of a light field energy surface in 4D image space according to a 4D plenoptic function. This figure shows ray traces of the energy surface 600 for an observer 620 illustrating how rays of energy converge in space 630 from various locations within the viewing volume. As shown, each waveguide element 610 defines four dimensions of information describing the propagation of energy 640 through the energy surface 600. The two spatial dimensions (referred to herein as x and y) are the physical plurality of energy positions as seen in image space, and the angular components theta and phi (herein referred to as referred to as u and v), which is observed in virtual space when projected through the energy waveguide array. In general and according to the 4D plenoptic function, a plurality of waveguides (e.g., lenslets) form a holographic or light field system described herein, forming a unique waveguide in virtual space from the x, y dimensions. The position allows the energy location to be directed along a direction defined by the u and v angle components.

However, significant challenges to light field and holographic display technologies include diffraction, scattering, diffusion, angular direction, calibration, focus, collimation, curvature, uniformity, element crosstalk, as well as reduced effective resolution and accuracy with sufficient fidelity. Those skilled in the art will understand that the inability to converge the energy results from uncontrolled propagation of energy due to designs that do not accurately account for any of the numerous other parameters that contribute to the inability to converge the energy.

In one embodiment, an approach to selective energy propagation to address challenges associated with holographic displays may include energy prohibition elements and waveguide openings with nearly collimated energy within an environment defined by a 4D plenoptic function. It may include substantial filling.

In one embodiment, an array of energy waveguides extends through the effective aperture of the waveguide element in unique directions defined by a defined 4D function, arranged to limit the propagation of each energy location to pass only through a single waveguide element. A plurality of energy propagation paths may be defined for each waveguide element configured to substantially populate a plurality of energy positions along a seamless energy surface that is inhibited by one or more elements.

In one embodiment, multiple energy domains may be configured within a single energy relay or between multiple energy waveguides to direct one or more sensory holographic energy propagations including visual, auditory, tactile or other energy domains.

In one embodiment, the energy waveguides and seamless energy surface are configured to receive and emit one or more energy domains to provide bidirectional energy propagation throughout the system.

In one embodiment, the energy waveguides are digitally encoded, diffractive, refractive, reflective, for orientation of any seamless energy surface, including walls, tables, floors, ceilings, rooms, or other geometry-based environments. It is configured to propagate non-linear or irregular distributions of energy, including non-transmitting void regions, utilizing waveguide configurations such as grin, holographic, Fresnel, etc. In additional embodiments, the energy waveguide elements are configured to create arbitrary surface profiles and/or various geometries that provide tabletop viewing, allowing the user to view holographic images anywhere around the energy surface in a 360-degree configuration. make it visible

In one embodiment, the energy waveguide array elements may be reflective surfaces, and the arrangement of the elements may be hexagonal, square, irregular, semi-regular, curved, non-planar, spherical, cylindrical, tilted, etc. ) may be regular, tilted, irregular, spatially variable, and/or multi-layered.

For any component within the seamless energy surface, waveguide, or relay components may include optical fibers, silicon, glass, polymers, optical relays, diffractive, holographic, refractive, or reflective elements, optical surface plates, energy couplers, beam splitters, etc. may include, but are not limited to, fields, prisms, polarizing elements, spatial light modulators, active pixels, liquid crystal cells, transparent displays, or similar materials that exhibit Andersen localization or total internal reflection.

Making the holodeck possible:

A collection of seamless energy surface systems that stimulate human sensory receptors within holographic environments

Seamless by tiling, fusing, bonding, attaching and/or stitching multiple seamless energy surfaces together to form arbitrary sizes, shapes, contours or form factors encompassing entire rooms. It is possible to build large-scale environments of energy surface systems. Each energy surface system may include an assembly having a base structure, energy surface, relays, waveguides, devices and electronics collectively configured for bi-directional holographic energy propagation, emission, reflection or detection.

In one embodiment, an environment of tiled seamless energy systems is aggregated to form large seamless planar or curved walls containing fixtures that include all surfaces in a given environment, including seamless, discrete planar, faceted, curved, and cylindrical surfaces. , consisting of any combination of spherical, geometric or irregular geometric structures.

In one embodiment, aggregated tiles of planar surfaces form wall-scale systems for stage- or venue-based holographic entertainment. In one embodiment, aggregated tiles of planar surfaces cover a room with four to six walls including both the ceiling and floor for cave-based holographic installations. In one embodiment, aggregated tiles of curved surfaces create a cylindrical seamless environment for immersive holographic installations. In one embodiment, aggregated tiles of seamless spherical surfaces form a holographic dome for immersive holodeck-based experiences.

In one embodiment, aggregated tiles of seamless curved energy waveguides provide mechanical edges that follow a precise pattern along the boundaries of the energy-barring elements within the energy waveguide structure to join, align, or otherwise align adjacent tiled mechanical edges of adjacent waveguide surfaces. The fusion results in a modular and seamless energy waveguide system.

In another embodiment of an aggregated tiling environment, energy propagates bidirectionally over multiple simultaneous energy domains. In a further embodiment, an energy surface provides the ability to simultaneously display and capture light field data from the same energy surface with waveguides designed such that light field data can be projected by an illumination source through the waveguide and simultaneously received through the same energy surface. In additional embodiments, additional depth sensing and active scanning technologies may be utilized to allow for interaction between energy propagation and an observer in a precise world coordinate system. In a further embodiment, the energy surface and waveguide are operable to radiate, reflect, or converge frequencies to induce a tactile sensation or volumetric haptic feedback. In some embodiments, any combination of bidirectional energy propagation and integrated surfaces is possible.

In one embodiment, the system utilizes one or more energy devices independently paired with two or more path energy couplers to enable pairing of at least two energy devices to the same portion of a seamless energy surface. comprising an energy waveguide capable of bi-directional emission and detection of energy, or one or more energy devices in front and outside the FOV of the waveguide for direct or reflected projection or sensing of an off-axis or on the base structure. It is fixed behind the energy surface in close proximity to an additional component fixed in position, the resulting energy surface having a two-way transmission of energy allowing the waveguide to converge the energy, a first device emitting the energy and a second device detecting the energy. Apparatus is provided, wherein the information includes detection of 4D plenoptic eye and retinal tracking or interference within propagated energy patterns, depth estimation, proximity, motion tracking, image, color or sound formation or other energy frequency analysis. It is processed to perform computer vision-related tasks, including but not limited to: In a further embodiment, the tracked positions actively calculate and modify positions of energy based on interference between bidirectional captured data and projection information.

In some embodiments, multiple combinations of three energy devices, including an ultrasonic sensor, a visible electromagnetic display, and an ultrasonic emitting device, comprise an energy domain of each device, and two engineered waveguides each configured for ultrasonic and electromagnetic energy. A separate energy domain configured together for each of the three first relay surfaces to propagate energy coupled to a single second energy relay surface with each of the three first surfaces comprising engineered properties specific to the elements. Provides the ability to direct and converge the energy of each device independently and substantially unaffected by other waveguide elements configured for it.

In some embodiments, calibrated configuration files as well as a calibration procedure to remove system artifacts and enable efficient manufacturing to generate a geometric mapping of the resulting energy surface for use with encoding/decoding techniques. Based on this, a dedicated integrated system for converting data into corrected information suitable for energy propagation is disclosed.

In some embodiments, a series of additional energy waveguides and one or more energy devices may be integrated into the system to create opaque holographic pixels.

In some embodiments, energy-inhibiting elements, beam splitters, prisms, active parallax barriers are used to provide spatial and/or angular resolution greater than the diameter of the waveguide or for other super-resolution purposes. Alternatively, additional waveguide elements containing polarization techniques may be incorporated.

In some embodiments, the disclosed energy system may also be configured as a wearable interactive device, such as virtual reality (VR) or augmented reality (AR). In other embodiments, the energy system may include steering optical element(s) that causes the displayed or received energy to be focused proximately to a determined plane in space for the viewer. In some embodiments, a waveguide array may be integrated into a holographic head mounted display. In other embodiments, the system may include multiple optical paths that allow an observer to view both the energy system and the real-world environment (eg, a transmissive holographic display). In this case, the system can be provided as a near field in addition to other methods.

In some embodiments, transmission of data includes encoding processes with selectable or variable compression ratios to receive any dataset of information and metadata; Analyze the dataset and receive or assign material properties, vectors, surface IDs, new pixel data to form a sparser dataset, and the received data includes two-dimensional, stereoscopic, multi-view, metadata, Contains a light field, hologram, geometry, vectors or vectorized metadata, and the encoder/decoder can be configured to: 2D; via depth estimation algorithms with or without depth metadata; 2D + depth, metadata or other vectorized information; stereoscopic, stereoscopic + depth, metadata or other vectorized information; multi view; Multi-view + depth, metadata or other vectorized information; hologram; Alternatively, it can provide the ability to transform data in real-time or offline, including image processing for light field content, and inverse ray tracing methods can provide a variety of 2D, stereoscopic, multi-view, and volumetric ray tracing methods via characterized 4D plenoptic functions. , appropriately maps the resulting transformation data generated by inverse ray tracing from light field or holographic data to real-world coordinates. In these embodiments, the desired overall data transmission may be several orders of magnitude less information transmitted than the raw light field dataset.

Overview of 4D plenoptic energy-directed system architecture

7 shows an overview of the architecture of one embodiment of a four-dimensional (4D) plenoptic energy directing system 700. Directed energy system 700 may include one or more energy devices 702, such as an LCD, LED, or OLDED, which may include energy locations 704 on a regular grid. Energy from energy location 704 can be directed to energy locations 712 on energy surface 706 via energy relay element 708, which is a tapered energy relay, flexible energy relay. It may include, but is not limited to, a relay or a face plate, each of which may operate according to the principle of Anderson transverse localization in some embodiments. Energy system 700 may include a mosaic of these energy devices 702 and relay elements 708. Each relay element 708 may introduce its own distortion 710, such as warping, such that the regular grid pattern on the energy device plane is no longer regular on the energy surface 706. On energy surface 706, energy system 700 may further include an array of energy waveguides 720. In one embodiment for visible electromagnetic energy, the array of energy waveguides 720 may be an array of lenses. Energy locations 704 within energy device 702 may have relayed energy locations 712 on energy surface 706 as defined by its (x, y) coordinates on the energy surface. The energy system 700 may further include an inhibition element 714 to inhibit the propagation of energy. Inhibition element 714 and energy waveguide array 720 may cooperate so that each of energy locations 712 has an uninhibited propagation path through the waveguide array location (x, y). The uninhibited propagation path of energy positions 712 can be characterized by a path angle defined by the angular coordinates (u, v), and the collimated energy wave is uninhibited in the angular coordinates (u, v). It can spread along the propagation path. Together, the four parameters (x, y, u, v) - the waveguide positions and the angular coordinates of the propagation paths through the waveguide elements - define a 4D plenoptic coordinate system. One of the goals of calibration is to determine the 4D parameters for each energy location 712 as accurately as possible.

Overview of the calibration procedure

FIG. 8 is a flow diagram illustrating one embodiment of a calibration process 800 that may include an energy location mapping process 802 and a waveguide mapping process 804. In one embodiment, the energy location mapping process 802 may include calibration for the relay element 708 without the waveguide 720. The energy location mapping process 802 may define actual coordinates (x, y) in physical space for each energy location 704 on the energy surface 706. The energy location mapping process 802 may also define a mapping between each energy location 712 on the energy surface 706 and each energy location 704 on the energy source device 702. In one embodiment, the energy location mapping process 802 may remove any distortion or artifacts 710 that the relay element 708 may introduce. Each individual energy relay in the field mosaic (referred to as a tile) is analyzed, and the overall mapping between the energy surface 706 and the corresponding energy device 702 for that tile is determined. During the fine step, smaller but more detailed parts of each tile are analyzed at once, and the mapping between the energy surface 706 and the energy device 702 is made much more accurate. In one embodiment, the energy location mapping process 802 also includes applying a gain map for each relay element tile, and adjusting the overall energy intensity of each relay element tile to match all others in the energy system 700. May include steps. In one embodiment, the waveguide array mapping process 804 of the calibration process 800 may be performed after the energy location mapping process 802 and after the energy waveguides 720 are aligned and secured. The waveguide array mapping process 804 may generally define an energy propagation path angle for energy propagating through each energy location under the energy waveguides 720. The waveguide mapping process 804 can produce accurate (u, v) angular coordinates for each energy location 712. In one embodiment, this calibration procedure places an energy location 712 below the center of each waveguide 720, assigns the energy locations 712, and finally assigns these (u, v) through direct measurement and interpolation. Verify and refine.

Overview of calibration settings for energy location

9 is a schematic diagram of a calibration system 900 for the energy location mapping process 802. In one embodiment, calibration system 900 may include energy sensor 902. Energy sensor 902 can be any device configured to receive energy from energy device 702 and relay element 708. For example, the energy sensor 902 may include a camera, a line scanning device, a plurality of pressure sensors arranged in a spatial array, or a plurality of acoustic sensors arranged in a spatial array. In one embodiment, energy sensor 902 may include a commercially available camera greater than 20 MPix that can be remotely operated via a computer for automated data collection. In one embodiment, the sensor size of the energy sensor 902 may be selected to be equal to the size of the energy surface 706 side of the individual relay element 708, with the number of pixels in each dimension in the horizontal plane being the energy location ( The number may be selected to be greater than 712 or 704). In one embodiment, the energy sensor may include a macro lens focused on the energy surface 706, providing an imaging field of view that is 10% larger than the energy surface 706 side of the individual relay elements 708. Thus, the entire relay element tile is imaged. The energy system 700 is mounted on a motorized movable platform 904 in a calibration system 900 that moves the energy system 700 below the energy sensor 902 in x and y coordinates parallel to the energy surface 706. It can be. In one embodiment, the calibration system 900 further includes an energy sensor 902 and a controller module 906 in electrical communication with the movable platform 904 so that the movable platform 904 can be remotely operated for automation. It can be controlled by the controller module 906 to move.

In one embodiment, the energy system 700 may be mounted on a tilt stage with two degrees of freedom and adjust the energy surface 706 to be coplanar with the plane of motion of the moveable platform 904. Allowed. The tilt can be adjusted by trial and error until the entire energy surface 706 remains in focus despite the shallow depth of field of the objective lens in one embodiment of the optics for the energy sensor 902.

The energy location mapping process 802 may allow mapping between the physical coordinates of energy locations 712 and the digital coordinates of energy locations 704 in the energy device 702.

One approach to do so is to first capture data with a first reference pattern placed on the energy surface 706 using the energy sensor 902 . The resulting data of the reference pattern resides in the digital space of the sensor with a known reference pattern in the plane of the energy surface 706. Figure 10A outlines the energy surface 706 for the energy relay recorded in the digital space of the sensor 902. Sensor data may include boundaries of specific relay mosaic tiles. Although the tile shown in Figure 10A is rectangular, the image of the tile may have subtle distortion, such as pin-cushion distortion. Figure 10b shows the actual coordinates of the energy surface 706 defined in physical reference space. Using the features of the reference pattern, it is possible to create a map between sensor coordinates defined in digital reference space and actual coordinates on surface 706. For convenience, this map is referred to herein as “Map 1.” In one embodiment, Map 1 converts digital sensor coordinates to real world coordinates in units of length (e.g., mm).

In one embodiment, after removing the first reference pattern, a second reference pattern may be present at energy locations 702 on the energy source device 702. This causes this second reference pattern to be relayed to the first surface 706. Relay element 708 may warp the second reference pattern as it is relayed to surface 706. Sensor data for this second reference pattern may be recorded in a digital reference space. At this point, Map 1 can be applied to the digital sensor coordinates to transform this second reference pattern into actual coordinates in physical reference space. In one embodiment, mapping the real coordinates of the surface 706 to the digital coordinates of the energy source plane 702 through feature detection and by knowing a second reference pattern present on the source device 702. This is possible, creating a map referred to herein as “Map 2”. In one embodiment, Map 2 converts the (x, y) coordinates of the energy surface 706 to digital coordinates of the energy source plane 702. This forward mapping has a paired reverse mapping. FIG. 10C shows the energy surface 706 in digital coordinates of the energy source plane 702 with contoured energy source plane coordinates mapping to physical edges of the surface of the relay 706.

FIG. 11 is a flow diagram illustrating one embodiment of an energy location mapping process 1100. The mapping process 1100 may include a step 1102 in which data is received about the energetic properties of the energy at the first plurality of energy locations 712 on the first surface 706 of the energy relay element 708. there is. Energy at the first plurality of energy locations 712 was relayed from the second plurality of energy locations 704 through the energy relay element 708 along the longitudinal orientation of the relay element 708. The mapping process 1100 may include predetermined data of the energy properties of the energy in the second plurality of energy locations 704 and data of the energy properties of the energy in the first plurality of energy locations 712 in the first plurality of energy locations 712. A step 1104 of correlating to generate a calibrated relay function comprising mapping the energy attributes at the energy positions 712 to the energy attributes at the second plurality of energies 704. . It should be understood that the energy properties correlated and mapped in step 1104 may include at least one energy property selected from the group consisting of position, color, intensity, frequency, amplitude, contrast, and resolution. In one embodiment, calibration mapping may be applied to compensate for at least one relay property selected from the group consisting of: intensity change, color change, attenuation area, and spatial distortion.

In one embodiment, the energy properties at the first plurality of energy locations 712 may include at least position coordinates defined in physical reference space, and the energy properties at the second plurality of energy locations 704 may include It may include at least position coordinates defined in the first digital reference space. For example, the first digital reference space may be defined by energy positions 704 in energy device 702. In one embodiment where energy device 702 includes a display, the pixels of the display may define a digital “pixel” reference space. In one embodiment, position coordinates defined in a physical reference space can be transformed from a second digital reference space using a transformation function. The second digital space may be defined by the sensing units of sensor 902. For example, data captured by sensor 902 may include captured pixel positions within the data, and a conversion function of the captured pixel positions to actual physical measurements may be used to convert the first plurality of energy positions from the physical measurement to a first plurality of energy positions. It can be used to transform the energy properties in fields 712.

In one embodiment, data characterizing the energy of the first plurality of energy locations 712 may be generated by an energy sensor 902 that captures energy from the first plurality of energy locations 712 . In one embodiment, energy sensor 902 may be configured to receive operating parameters of energy sensor 902 from a controller 906, which may be programmed to operate energy sensor 902 according to predetermined commands. there is. In one embodiment, operating parameters may be provided as digital signals from controller 906. In one embodiment, the operating parameters may include position commands by which the controller 906 is programmed to position the sensor 902 according to predetermined commands.

In one embodiment, data of the nature of the energy at the first plurality of energy locations 712 is obtained by positioning the movable platform 904 on which the energy relay element 708 is positioned and the energy relay element 708 may be generated by activating the energy sensor 902 to capture energy from the first plurality of energy locations 712 when positioned at a predetermined location. In one embodiment, the movable platform 904 and sensors 902 receive digital signals from a controller 906 that can be programmed to operate the energy sensors 902 and movable platform 904 according to predetermined commands. It is configured to receive signals. In one embodiment, the digital signals may include position commands for the energy sensor 902 and the movable platform 904, and the controller 906 may control the energy sensor 902 and the movable platform 904 according to predetermined commands. 904) is programmed to position.

FIG. 12 is a flow diagram illustrating one embodiment of an energy location mapping process 1200 that converts energy attributes in digital space to energy attributes in physical space using a transformation function. The mapping process 1200 includes the steps 1206 in which data regarding capture reference energy properties of the reference energy captured at the first plurality of energy locations 712 on the first surface 706 of the energy relay element 708 are received ( 1206 ). ) may include. The reference energy may have predetermined reference energy properties defined in a physical reference space, for example actual measurements. The predetermined reference energy attribute may include at least one energy attribute selected from the group consisting of position, color, intensity, frequency, amplitude, contrast, and resolution. In one embodiment, the reference energy forms a reference spatial pattern, wherein the reference energy captured at the first plurality of energy positions 712 on the first surface 706 forms a capture pattern. In one embodiment, the positional properties of the reference space pattern are known in physical reference space.

In one embodiment, the mapping process 1200 may include a step 1208 in which a transformation function is generated by correlating predetermined reference energy properties defined in a physical reference space with reference energy properties captured in a digital reference space. In one embodiment, this mapping is similar to Map 1 above, with image pixel coordinates converted to real coordinates in units of length (e.g., mm), and is the mapping shown between FIGS. 10A and 10B.

The mapping process 1200 includes a step 1202 in which digital data is received about the energetic properties of the energy at the first plurality of energy locations 712 on the first surface 706 of the energy relay element 708. can do. Energy at the first plurality of energy locations 712 was relayed from the second plurality of energy locations 704 through the energy relay element 708 along the longitudinal orientation of the relay element 708. The mapping process 1200 applies the transformation function generated in step 1208 to the digital data received in step 1202 to map the energy properties of the energy defined in the digital reference space to the energy properties in the physical reference space. A step 1204 may be further included.

In one embodiment, the mapping process 1200 combines an energy attribute at a first plurality of energy locations 712 defined in a physical reference space with an energy attribute at a second plurality of energy locations 704 defined in a digital reference space. A step 1210 of generating a mapping between energy properties may be further included. In one embodiment, this mapping is similar to Map 2 above, which converts real-world coordinates in length units (e.g., mm) to energy source digital pixel coordinates, and is the mapping shown between FIGS. 10B and 10C.

Illustrative Implementation 1

To illustrate the principles of the invention, an example implementing embodiments of mapping processes 1100 and 1200 is shown below in the context of a display system having pixels of a display surface and illumination sources capable of providing an image to the display surface. provided. It should be understood that other implementations according to the principles of the present invention may be performed with other types of energy systems, such as acoustic, infrared, ultraviolet, microwave, x-ray, electromagnetic, opto-mechanical, or tactile energy systems.

1. Place a high-resolution reference checkerboard chart directly on the display surface. This is a reference grid with a known pitch used to calibrate real-world coordinates, and is printed on a transparent medium such as Mylar or glass. The dark areas of the chart must be optically opaque, and the bright areas of the chart must be optically transparent. If the checkerboard chart adds arbitrary path length, for example from the thickness of the glass, the corrected path length must also be included during imaging of the display surface without this chart. In at least one embodiment, the pitch of the reference checkerboard is 125um.

2. Capture an image of the chart for each tile of the optical relay mosaic, or if there are no tiles, there is some neighboring frame overlap for each part of the display matching the camera's FOV. The checkerboard chart should be backlit by uniformly illuminating the energy source plane. The display is moved below the camera by the moving stage shown in Figure 2.

3. Identify the reference checkerboard grid pattern.

4. Create map 1 from image space to real coordinates. This calibrates the distance and eliminates distortions due to lens or imperfect camera alignment. This mapping should be applied to all calibration images after this point.

5. Remove the high-resolution checkerboard chart placed on the display surface.

6. Capture a white image of a single tile of the optical relay mosaic while uniformly illuminating.

7. Perform edge detection on this white image to determine the boundaries of the optical relay tiles.

8. Calculate the rotation of the optical relay tiles in the image, and apply a reverse rotation to ensure that the optical relay tile boundaries are not rotated with the image boundaries, but rather have straight optical relay tile boundaries.

9. Place a known checkerboard pattern on the energy source plane. In at least one embodiment, the checkerboard squares are each 4 to 12 pixels wide. The checkerboard pattern on the surfaced display, if present, may be optically distorted after passing through the optical relay.

10. Capture the image of the optical relay tile and segment it pixel by pixel with a white image to remove (normalize) any blemishes or intensity variations through the optical relay.

11. Apply Map 1 to convert this image to real coordinates.

12. Identify the grid pattern of the checkerboard image, and the boundaries of the optical mosaic tiles.

13. Apply Map1 to this image to determine the actual coordinates of the grid pattern on the display surface.

14. Determine the mapping map 2 from display surface real world coordinates to lighting engine pixel coordinates. This mapping may use at least one offset pixel reference location for the illumination source display relative to a known location on the display surface, such as the top-left corner.

15. Two mappings are now identified: Map 1 and Map 2.

16. If the display surface consists of optical relay tiles, each tile must be imaged as a uniform white image. The average illumination can then be determined, and the overall illumination of each tile can be adjusted to achieve a uniform display surface.

Overview of 4D Calibration

The energy location calibration described above may define a mapping between the actual coordinates for the energy surface 706 and the coordinates for the energy locations 704 within the energy device 702. This energy position calibration can be performed without the energy waveguide array 720. 4D calibration can define angular coordinates for the energy propagation path for each energy location 712 once the waveguide 720 is installed. The purpose of the 4D calibration process is to define 4D plenoptic coordinates (x, y, u, v) for every energy location 712.

13 and 14, in one embodiment, energy waveguide element 720a has uninhibited energy propagation paths 1402, 1404 extending from a first side to a second side of energy waveguide element 720a. It may be operable to direct energy along . Uninhibited energy propagation paths 1402 and 1404 may extend to a plurality of energy locations 1406 and 1408, respectively, on a first side of energy relay element 706, each energy location on the first side Depending on the energy waveguide element 720a, fields 1406 and 1408 may extend along different angular directions (u ₁ , v ₁ ), (u ₂ , v ₂ ) on the second side.

In one embodiment, the process 1300 for 4D calibration includes the step of receiving 1302 data about energy properties of energy along uninhibited energy propagation paths 1402, 1404 on the second side of the waveguide array. and correlating energy property data of energy at a plurality of energy locations (1406, 1408) on the first side of the energy waveguide element with energy property data of energy along uninhibited energy propagation paths (1402 and 1404) on the second side. generating a calibrated four-dimensional (4D) plenoptic function for the waveguide element 720a (step 1304). The calibrated 4D plenoptic function may include a mapping between a plurality of energy locations 1406 and 1408 and respective angular directions of the energy propagation paths 1402 and 1404.

In one embodiment, the data of the data attribute of energy along the non-inhibited energy propagation paths 1402, 1404 on the second side of the waveguide array include the data of the data attribute of the energy along the non-inhibited energy propagation paths 1402, 1404 on the second side of the waveguide array. It may be generated by an energy sensor 902 that captures energy along 1404). In one embodiment, energy sensor 902 may be configured to receive operating parameters of energy sensor 902 from a controller 906, which may be programmed to operate energy sensor 902 according to predetermined commands. . In one embodiment, operating parameters may be provided in digital signals from controller 906. In one embodiment, the operating parameters may include position commands by which the controller 906 is programmed to position the sensor 902 according to predetermined commands.

In one embodiment, data of the energy properties of energy along the uninhibited energy propagation paths 1402, 1404 on the second side of the waveguide array are used to position the movable platform 904 on which the energy relay element 708 is located. an energy sensor 902 to capture energy along the uninhibited energy paths 1402, 1404 on the second side of the waveguide array when the energy relay element 708 is positioned at a predetermined position. It can be created by operating . In one embodiment, the movable platform 904 and sensors 902 receive digital signals from a controller 906 that can be programmed to operate the energy sensors 902 and movable platform 904 according to predetermined commands. It is configured to receive signals. In one embodiment, the digital signals may include position commands for the energy sensor 902 and the movable platform 904, and the controller 906 may control the energy sensor 902 and the movable platform 904 according to predetermined commands. 904) is programmed to position.

It should be understood that the energy properties correlated and mapped in step 1304 may include at least one energy property selected from the group consisting of position, color, intensity, frequency, amplitude, contrast, and resolution. In one embodiment, a calibrated 4D plenoptic function may be applied to compensate for at least one waveguide array property selected from the group consisting of: intensity variation, color variation, attenuation area, and spatial distortion. In one embodiment, process 1100 or 1200 may be performed to compensate for at least one relay property, followed by process 1300 to compensate for at least one waveguide array property, thereby providing overall energy direction. System 700 may be compensated.

In one embodiment, the energy nature of the energy along the uninhibited energy propagation paths 1402, 1404 on the second side of the energy waveguide element 720a is at least the angle of the uninhibited energy propagation paths 1402, 1404. The energy properties of the energy at the plurality of energy positions 1406, 1408 on one side of the energy waveguide element 720a include the coordinates (u ₁ , v ₁ ), (u ₂ , v ₂ ), and the energy attribute of the energy is the plurality of energies. Contains at least the location coordinates of the locations. In one embodiment, the position coordinates for the plurality of energy locations 1406, 1408 are defined in physical reference space using a transformation function as described above with respect to processes 1100 and 1200, or are defined in physical reference space in digital reference space. It can be converted to reference space.

In one embodiment, the energy location mapping process of 1100 is performed prior to process 1300 such that the location coordinates for the plurality of energy locations 1406, 1408 are aligned with each of the uninhibited energy propagation paths 1402 in physical reference space. , 1404) can be used to determine the angular coordinates (u ₁ , v ₁ ), (u ₂ , v ₂ ). In one embodiment, the angular coordinates (u ₁ , v ₁ ), (u ₂ , v ₂ ) of the uninhibited energy propagation paths 1402 and 1404 in physical reference space are the known coordinates of the energy waveguide element 720a. It can be determined using the reference position, the known distance 1410 between the sensor 902 and the waveguide element 720a, and the known location of the reference energy location 712a, all of which are defined in the same physical reference space. For example, an energy propagation axis 1412 can be defined in waveguide element 720a and used as a reference position. The distance 1410 between sensor 902 and waveguide element 720a can be determined according to a number of measurement methods known in the art. Given these known reference parameters in physical space, the angular coordinates (u ₁ , v ₁ ), (u ₂ , v ₂ ) of the uninhibited energy propagation paths 1402, 1404 are determined by the known reference parameters in physical reference space. The parameters may be determined using triangulation of data points by sensor 902.

Illustrative Implementation 2

To illustrate the principles of the present disclosure, an example implementing embodiments of mapping 1300 may be used to image a display surface and a waveguide array, e.g., to a microlens that directs light from the display surface for viewing according to a 4D preoptic function. Provided below is a display system having pixels of an illumination source capable of providing . It should be understood that other implementations according to the principles of the present disclosure may be performed with other types of energy systems, such as acoustic, infrared, ultraviolet, microwave, x-ray, electromagnetic, opto-mechanical, or tactile energy systems.

1. Measure the distance between the known vertical reference laser beam and the corner of the display.

2. Determine the lens positions to be measured. This may be regular sampling across the display surface.

3. Move the display so that the camera is directly over the center of each lens by instructing the moving stage using the known lens position geometry.

4. Illuminate each pixel under the lens. For each, measure the beam position of the camera sensor.

5. Using this beam position and the known height of the sensor above the lens, determine the angle the beam makes with the vertical z axis in the x-z plane () and y-z plane (). These angles can be converted to normalized (u, v) coordinates.

6. Repeat this for multiple pixels under the lens.

7. Repeat this procedure to regularly sample the disks across the display surface.

8. Find the (u, v) coordinates as a function of the pixel offset from the disk center, and fit a transformation of these parameters to a polynomial in each axis along the width and height of the entire display surface.

While various embodiments have been described above in accordance with the principles disclosed herein, it should be understood that they are presented by way of example only and not by way of limitation. Accordingly, the breadth and scope of the present invention(s) should not be limited by any of the foregoing exemplary embodiments, but should be defined only in accordance with the claims and their equivalents issued from this disclosure. Additionally, while the foregoing advantages and features are provided in the described embodiments, the application of these published claims should not be limited to processes and structures that achieve any or all of the above advantages.

It will be understood that the key features of the present disclosure may be employed in various embodiments without departing from the scope of the invention. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific procedures described herein. Such equivalents are considered to be within the scope of this disclosure and are covered by the claims.

Additionally, section headings herein are provided for consistency with the suggestions under 37 CFR 1.77 or otherwise to provide structural clues. These headings are not intended to limit or characterize the invention(s) described in any claims that may emerge from this disclosure. Specifically, by way of example, although the headings refer to "fields of invention," such claims should not be limited by the language under this heading, but rather describe so-called technical fields. Additionally, the description of the technology in the “Background of the Invention” section should not be construed as an admission that the technology is prior art to any invention(s) in this disclosure. Additionally, the “Summary” should not be considered a characterization of the invention(s) specified in the published claims. Moreover, any reference to “invention” in this disclosure should not be used to argue that only a single point of novelty exists in this disclosure. Numerous inventions may be specified within the scope of the multiple claims issued from this disclosure, such claims thereby defining the invention(s) protected thereby and equivalents thereof. In all cases, the scope of such claims should be considered on their own merits in light of this disclosure but should not be limited by the headings specified herein.

The use of the term one or "an", when used in conjunction with the term "comprising" in the claims and/or specification, may mean "one", but also "one or more", "at least one", and It is consistent with the meaning of “one or more than one.” Although the present disclosure supports definitions that refer only to the alternatives and "and/or," the use of the term "or" in the claims explicitly indicates that it refers only to the alternatives or when the alternatives are mutually exclusive. Except, it is used to mean “and/or”. Throughout this application, the term "about" is used to indicate that a value includes the inherent variation in error for the device and method employed to determine the value, or the variation that exists between study subjects. In general, and subject to the foregoing discussion, numerical values herein as modified by words of approximation such as “about” are at least ±1, 2, 3, 4, 5, 6, 7, 10, 12 from the stated value. Or it can change by 15%.

As used in the specification and claim(s), “comprising” (and any form of including, such as “comprises” and “comprises”), “having” (and any form of having, such as “to have,” “to have,” “to include,” (and any form of containing, such as “including” and “including”), or “containing” (and any form of containing, such as “containing”) “Does” and “contains”) are inclusive or open-ended and do not exclude additional unstated elements or method steps.

Words of comparison, measurement, and timing, such as “at the time,” “equivalent,” “during,” “done,” etc., are used to refer to “substantially at the time,” “substantially equivalent,” “substantially en route,” “ “substantially completed,” etc., where “substantially” means that such comparisons, measurements, and timing are practicable to achieve the desired results, implicitly or explicitly stated. Words relating to the relative position of elements, such as nearby, "nearby" and "adjacent", mean close enough to have a physical effect on the interaction of each system element. Other words for approximation similarly refer to conditions that are not necessarily absolute or complete when so modified, but are understood to be sufficiently close to those of ordinary skill in the art to warrant considering those conditions as existing. The extent to which the description can be changed will depend on how large the change can be introduced, yet those skilled in the art will recognize the modified feature while still retaining the desired characteristics and capabilities of the unmodified feature.

As used herein, the term “or combination thereof” refers to all permutations and combinations of the listed items preceding the term. For example, A, B, C, or any combination thereof, is intended to include at least one of A, B, C, AB, AC, BC, or ABC, or, if the order is important in a particular context, BA, CA, CB, CBA , is intended to include BCA, ACB, BAC or CAB. Continuing with this example, combinations containing repeated occurrences of one or more items or terms are explicitly included, such as BB, AAA, AB, BBC, AAABCCCC, CBBAAA, CABABB, etc. One of ordinary skill in the art will understand that there is no limitation on the number of items or terms in any combination unless otherwise obvious from the context.

All structures and/or methods disclosed and claimed herein can be made and practiced without undue experimentation in light of the present disclosure. Although the configurations and methods of the present disclosure have been described in terms of preferred embodiments, modifications may be made to the configurations and/or methods and steps of the methods described herein without departing from the concept, spirit, and scope of the disclosure. Alternatively, it will be clear to a person skilled in the art that it can be applied in a sequence of steps. All such similar alternatives and modifications apparent to those skilled in the art are deemed to be within the spirit, scope and concept of the disclosure as defined by the appended claims.

Claims (36)

A calibration method for an energy relay element, comprising:
The energy relay element is configured so that energy propagating through the energy relay element has higher transmission efficiency in the longitudinal direction,
The above method is,
Receiving data of energy properties of energy at a first plurality of energy locations on a first surface of an energy relay element, wherein the energy at the first plurality of energy locations is directed to the energy relay element along the longitudinal direction. the receiving step, wherein the energy is relayed from a second plurality of locations; and
Correlating predetermined data of energy properties of energy in the second plurality of energy locations with data of energy properties of energy in the first plurality of energy locations to generate a calibrated relay function,
The method of claim 1, wherein the calibrated relay function includes a mapping of the energy properties at the first plurality of energy locations to the energy properties at the second plurality of energy locations. According to paragraph 1,
The energy properties at the first plurality of energy locations include at least one position coordinate defined in a physical reference space, and the energy properties at the second plurality of energy locations include at least a position coordinate defined in a first digital reference space. How to include it. According to paragraph 2,
A method in which position coordinates defined in the physical reference space are converted from a second digital reference space. According to paragraph 1,
The above method is,
Receiving data of captured reference energy properties of a reference energy captured at the first plurality of energy locations on the first surface of the energy relay element, wherein the reference energy is a predetermined reference defined in physical reference space. the receiving step having energy properties;
Correlating the predetermined reference energy properties with the captured reference energy properties to generate a transformation function; and
further comprising applying the transformation function to energy properties of the energy at the first plurality of energy locations to map energy properties of the energy defined in a digital reference system to energy properties of the energy in the physical reference space. Including,
the mapping of the calibrated relay function maps energy properties at the first plurality of energy locations to energy properties at the second plurality of energies,
The method of claim 1 , wherein energy properties at the first plurality of energy locations are defined in the physical reference space and energy properties at the second plurality of energies are defined in the digital reference space. According to paragraph 4,
The method of claim 1, wherein the predetermined reference energy properties include at least one energy property selected from the group consisting of position, color, intensity, frequency, amplitude, contrast and resolution. According to paragraph 4,
The method of claim 1, wherein the reference energy forms a reference spatial pattern, and the reference energy captured at the first plurality of energy locations on the first surface forms a captured pattern. According to clause 5,
Wherein the positional properties of the reference space pattern are known in the physical reference space. According to paragraph 1,
A method wherein data of energy properties of energy at the first plurality of energy locations is generated by an energy sensor that captures energy from the first plurality of energy locations. According to clause 8,
The method of claim 1, wherein the energy sensor includes a camera, a line scanning device, a plurality of pressure sensors disposed in a spatial array, or a plurality of acoustic sensors disposed in a spatial array. According to clause 8,
The method of claim 1, wherein the sensor is configured to receive operating parameters of the energy sensor from a controller, and the controller is programmed to operate the energy sensor according to predetermined instructions. According to clause 10,
The method of claim 1, wherein the operating parameters are provided as digital signals from the controller. According to clause 11,
The method of claim 1, wherein the operating parameters include position instructions, and the controller is programmed to position the sensor according to the predetermined instructions. According to clause 8,
Data of energy properties of energy at the first plurality of energy locations are obtained by positioning a movable platform on which the energy relay element is positioned, and when the energy relay element is positioned at a predetermined location. A method created by operating the energy sensor to capture energy from. According to clause 13,
The method of claim 1 , wherein the movable platform and the energy sensor are configured to receive digital signals from a controller, wherein the controller is programmed to operate the energy sensor and the movable platform according to the predetermined instructions. According to clause 14,
The method of claim 1, wherein the digital signal includes positioning instructions for the energy sensor and the movable platform, and the controller is programmed to position the energy sensor and the movable platform according to the predetermined instructions. According to paragraph 1,
The method of claim 1 , wherein the energy properties of the energy at the first plurality of energy locations include at least one energy property selected from the group consisting of location, color, intensity, frequency, amplitude, contrast, and resolution. According to paragraph 1,
The method of claim 1 , wherein the energy properties of the energy at the second plurality of energy locations include at least one energy property selected from the group consisting of location, color, intensity, frequency, amplitude, contrast, and resolution. According to paragraph 1,
The calibration mapping is applied to compensate for at least one relay property selected from the group consisting of intensity change, color change, attenuation area, and spatial distortion. A calibration method for an energy waveguide array, comprising:
The energy waveguide array is operable to direct energy along uninhibited energy propagation paths extending from a first side to a second side of the energy waveguide array, wherein the uninhibited energy propagation paths extend from the first side to the second side of the energy waveguide array. extending to a plurality of energy positions on a plane, and extending on the second side along a different angular direction relative to the energy waveguide array depending on each energy position on the first side;
The above method is,
receiving data of energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array; and
Calibrating the energy waveguide array by correlating data of energy properties of energy at the plurality of energy locations with data of energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array. Including generating a four-dimensional (4D) plenoptic function;
The method of claim 1 , wherein the calibrated 4D plenoptic function includes a mapping between the plurality of energy locations and respective angular directions of the uninhibited energy propagation paths. According to clause 19,
Data on energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array may be configured to allow an energy sensor to capture energy along the uninhibited energy propagation paths on the second side of the waveguide array. How it is created by doing. According to clause 20,
The method of claim 1, wherein the energy sensor includes a camera, a line scanning device, a plurality of pressure sensors disposed in a spatial array, or a plurality of acoustic sensors disposed in a spatial array. According to clause 20,
The energy sensor is configured to receive operating parameters of the energy sensor from a controller, and the controller is programmed to operate the energy sensor according to a predetermined command. According to clause 22,
A method wherein the operating parameters are provided as digital signals from the controller. According to clause 23,
The method of claim 1, wherein the operating parameters include position instructions, and the controller is programmed to position the sensor according to the predetermined instructions. According to clause 20,
Data of energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array are obtained by positioning a movable platform on which the energy relay element is located, and when the energy waveguide array is positioned at a predetermined position. . According to clause 25,
The method of claim 1 , wherein the movable platform and the energy sensor are configured to receive a digital signal from a controller, and the controller is programmed to operate the energy sensor and the movable platform according to predetermined instructions. According to clause 26,
The method of claim 1, wherein the digital signal includes positioning instructions for the energy sensor and the movable platform, and the controller is programmed to position the energy sensor and the movable platform according to the predetermined instructions. According to clause 19,
Energy properties of energy along the uninhibited energy propagation paths on the second side of the energy waveguide array include at least an angular coordinate of the uninhibited energy propagation paths,
The method of claim 1 , wherein energy properties of energy at the plurality of energy locations on the first side of the energy waveguide array include at least position coordinates of the plurality of energy locations. According to clause 20,
A method in which the location coordinates are defined in a physical reference space. According to clause 29,
A method in which the position coordinates are converted from a digital reference space to a physical reference space using a conversion function. According to clause 20,
A method in which the angular coordinates are defined in physical reference space. According to clause 31,
Data on energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array may be configured to allow an energy sensor to detect energy along the uninhibited energy propagation paths on the second side of the waveguide array. Created by capturing,
The angular coordinates defined in the physical reference space are determined using a known reference position of the energy waveguide array, a known distance between the energy sensor and the waveguide array, and a known position of the reference energy position,
A method wherein the known reference location of the energy waveguide array, the known distance between the energy sensor and the waveguide array, and the known location of the reference energy location are all defined in the same physical reference space. According to clause 19,
The method of claim 1 , wherein the energy properties of the energy at the plurality of energy locations include at least one energy property selected from the group consisting of location, color, intensity, frequency, amplitude, contrast, and resolution. According to clause 19,
The energy properties of the energy along the uninhibited energy propagation paths on the second side of the energy waveguide array include at least one energy property selected from the group consisting of position, color, intensity, frequency, amplitude, contrast and resolution. How to. According to clause 19,
The calibration mapping is applied to compensate for at least one relay property selected from the group consisting of intensity change, color change, attenuation area, and spatial distortion. A calibration method for an energy-directed system, comprising:
The energy relay element of the energy directing system is configured so that energy propagating through the energy relay element has higher transmission efficiency in the longitudinal direction,
wherein the energy waveguide array of the energy directing system is operable to direct energy along uninhibited energy propagation paths extending from a first side to a second side of the energy waveguide array,
The uninhibited energy propagation paths extend to a plurality of relayed energy locations on the first side and extend to the second side along a different angular direction relative to the energy waveguide array depending on each energy location on the first side. extends from above,
The above method is,
Receiving data of energy properties of energy at a plurality of relayed energy locations on a first surface of the energy relay element, wherein energy at the first plurality of energy locations passes through the energy relay element along the longitudinal direction. receiving, relayed from a plurality of source energy locations;
Correlating predetermined data of energy properties of energy at the plurality of source energy locations with data of energy properties of energy at the plurality of relayed energy locations to generate a calibrated relay function, the calibrated relay function comprises a mapping of the energy properties at the first plurality of energy positions to the energy properties at the second plurality of energies,
receiving data of energy properties of energy along the uninhibited energy propagation paths on a second side of the waveguide array; and
for the energy waveguide array by correlating data of energy properties of energy at the plurality of relayed energy locations with data of energy properties of energy along the uninhibited energy propagation paths on the second side of the waveguide array. generating a calibrated four-dimensional (4D) plenoptic function, wherein the calibrated 4D plenoptic function includes a mapping between the plurality of energy positions and respective angular directions of the uninhibited energy propagation paths. A method comprising generating a calibrated 4D plenoptic function.