More Web Proxy on the site http://driver.im/

research-article

Optimal presentation of imagery with focus cues on multi-plane displays

Authors:

Rachel A. Albert,

Abdullah Bulbul,

Gregory J. Ward,

Martin S. Banks,

James F. O'BrienAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 34, Issue 4

Article No.: 59, Pages 1 - 12

https://doi.org/10.1145/2766909

Published: 27 July 2015 Publication History

Abstract

We present a technique for displaying three-dimensional imagery of general scenes with nearly correct focus cues on multi-plane displays. These displays present an additive combination of images at a discrete set of optical distances, allowing the viewer to focus at different distances in the simulated scene. Our proposed technique extends the capabilities of multi-plane displays to general scenes with occlusions and non-Lambertian effects by using a model of defocus in the eye of the viewer. Requiring no explicit knowledge of the scene geometry, our technique uses an optimization algorithm to compute the images to be displayed on the presentation planes so that the retinal images when accommodating to different distances match the corresponding retinal images of the input scene as closely as possible. We demonstrate the utility of the technique using imagery acquired from both synthetic and real-world scenes, and analyze the system's characteristics including bounds on achievable resolution.

Supplementary Material

ZIP File (a59-narain.zip)

Supplemental files

Download
61.80 MB

MP4 File (a59.mp4)

Download
16.42 MB

References

[1]

Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S. 2004. A stereo display prototype with multiple focal distances. ACM Trans. Graph. 23, 3 (Aug.), 804--813.

Digital Library

[2]

Andersen, A., and Kak, A. 1984. Simultaneous algebraic reconstruction technique (SART): A superior implementation of the ART algorithm. Ultrasonic Imaging 6, 1, 81--94.

[3]

Banks, M. S., Bulbul, A., Albert, R. A., Narain, R., O'Brien, J. F., and Ward, G. 2014. The perception of surface material from disparity and focus cues. In Proc. Vision Sciences Society 14th Annual Meeting.

[4]

Buckley, D., and Frisby, J. P. 1993. Interaction of stereo, texture and outline cues in the shape perception of three-dimensional ridges. Vision Research 33, 7, 919--933.

[5]

Campbell, F. 1957. The depth of field of the human eye. Optica Acta: International Journal of Optics 4, 4, 157--164.

[6]

Chambolle, A., and Pock, T. 2011. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40, 1, 120--145.

Digital Library

[7]

Coelho, J. M. P., Baião, A., and Vieira, P. 2013. Development of an optical simulator of the human eye. Proc. SPIE 8785, 8785CS--8785CS--8.

[8]

Cossairt, O. S., Napoli, J., Hill, S. L., Dorval, R. K., and Favalora, G. E. 2007. Occlusion-capable multiview volumetric three-dimensional display. Appl. Opt. 46, 8 (Mar), 1244--1250.

[9]

Du, S.-P., Masia, B., Hu, S.-M., and Gutierrez, D. 2013. A metric of visual comfort for stereoscopic motion. ACM Trans. Graph. 32, 6 (Nov.), 222:1--222:9.

Digital Library

[10]

Emoto, M., Niida, T., and Okano, F. 2005. Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television. Display Technology, Journal of 1, 2, 328--340.

[11]

Esser, E., Zhang, X., and Chan, T. 2010. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM Journal on Imaging Sciences 3, 4, 1015--1046.

Digital Library

[12]

Favalora, G. E., Napoli, J., Hall, D. M., Dorval, R. K., Giovinco, M., Richmond, M. J., and Chun, W. S. 2002. 100-million-voxel volumetric display. In Proc. SPIE, vol. 4712, 300--312.

[13]

Granger, E., and Cupery, K. 1972. Optical merit function (SQF), which correlates with subjective image judgments. Photographic Science and Engineering 16, 3.

[14]

He, B., and Yuan, X. 2012. Convergence analysis of primal-dual algorithms for a saddle-point problem: From contraction perspective. SIAM Journal on Imaging Sciences 5, 1, 119--149.

Digital Library

[15]

Held, R. T., Cooper, E. A., O'Brien, J. F., and Banks, M. S. 2010. Using blur to affect perceived distance and size. ACM Transactions on Graphics 29, 2 (Mar.), 19:1--16.

Digital Library

[16]

Hirsch, M., and Lanman, D., 2010. Build your own 3D display. ACM SIGGRAPH ASIA Course Notes.

Digital Library

[17]

Hoffman, D. M., Girshick, A. R., Akeley, K., and Banks, M. S. 2008. Vergence--accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision 8, 3.

[18]

Hu, X., and Hua, H. 2013. An optical see-through multi-focal-plane stereoscopic display prototype enabling nearly correct focus cues. vol. 8648, 86481A.

[19]

Hu, X., and Hua, H. 2014. Design and assessment of a depth-fused multi-focal-plane display prototype. J. Display Technol. 10, 4 (Apr), 308--316.

[20]

Hu, X., and Hua, H. 2014. High-resolution optical see-through multi-focal-plane head-mounted display using freeform optics. Opt. Express 22, 11 (Jun), 13896--13903.

[21]

Huang, F.-C., Lanman, D., Barsky, B. A., and Raskar, R. 2012. Correcting for optical aberrations using multilayer displays. ACM Transaction on Graphics 31 (Nov.).

Digital Library

[22]

Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R. 2014. Eyeglasses-free display: Towards correcting visual aberrations with computational light field displays. ACM Trans. Graph. 33, 4 (July), 59:1--59:12.

Digital Library

[23]

Ives, F., 1903. Parallax stereogram and process of making same., Apr. 14. US Patent 725,567.

[24]

Jakob, W., 2010. Mitsuba renderer. http://www.mitsuba-renderer.org.

[25]

Jones, A., McDowall, I., Yamada, H., Bolas, M., and Debevec, P. 2007. Rendering for an interactive 360° light field display. ACM Trans. Graph. 26, 3 (July).

Digital Library

[26]

Lambooij, M., Fortuin, M., Heynderickx, I., and Ijsselsteijn, W. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. Journal of Imaging Science and Technology 53, 3, 30201-1--30201-14.

[27]

Lang, M., Hornung, A., Wang, O., Poulakos, S., Smolic, A., and Gross, M. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graph. 29, 4 (July), 75:1--75:10.

Digital Library

[28]

Lanman, D., and Luebke, D. 2013. Near-eye light field displays. ACM Trans. Graph. 32, 6 (Nov.), 220:1--220:10.

Digital Library

[29]

Lanman, D., Hirsch, M., Kim, Y., and Raskar, R. 2010. Content-adaptive parallax barriers: Optimizing dual-layer 3D displays using low-rank light field factorization. ACM Trans. Graph. 29, 6 (Dec.), 163:1--163:10.

Digital Library

[30]

Lanman, D., Wetzstein, G., Hirsch, M., Heidrich, W., and Raskar, R. 2011. Polarization fields: Dynamic light field display using multi-layer LCDs. ACM Trans. Graph. 30, 6.

Digital Library

[31]

Levoy, M., Ng, R., Adams, A., Footer, M., and Horowitz, M. 2006. Light field microscopy. ACM Trans. Graph. 25, 3 (July), 924--934.

Digital Library

[32]

Lippmann, G. 1908. Épreuves réversibles donnant la sensation du relief. J. Phys. Theor. Appl. 7, 1, 821--825.

[33]

Liu, S., and Hua, H. 2010. A systematic method for designing depth-fused multi-focal plane three-dimensional displays. Opt. Express 18, 11 (May), 11562--11573.

[34]

Liu, S., Hua, H., and Cheng, D. 2010. A novel prototype for an optical see-through head-mounted display with addressable focus cues. IEEE Transactions on Visualization and Computer Graphics 16, 3, 381--393.

Digital Library

[35]

Love, G. D., Hoffman, D. M., Hands, P. J., Gao, J., Kirby, A. K., and Banks, M. S. 2009. High-speed switchable lens enables the development of a volumetric stereoscopic display. Opt. Express 17, 18 (Aug), 15716--15725.

[36]

MacKenzie, K. J., Hoffman, D. M., and Watt, S. J. 2010. Accommodation to multiple-focal-plane displays: Implications for improving stereoscopic displays and for accommodation control. Journal of Vision 10, 8.

[37]

MacKenzie, K., Dickson, R., and Watt, S. 2012. Vergence and accommodation to multiple-image-plane stereoscopic displays: "real world" responses with practical image-plane separations? Journal of Electronic Imaging 21, 1.

[38]

Maimone, A., Wetzstein, G., Lanman, D., Hirsch, M., Raskar, R., and Fuchs, H. 2013. Focus 3D: Compressive accommodation display. ACM Trans. Graph. 32, 5, 1--13.

Digital Library

[39]

Mantiuk, R., Kim, K. J., Rempel, A. G., and Heidrich, W. 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30, 4 (July), 40:1--40:14.

Digital Library

[40]

Marshall, J. A., Burbeck, C. A., Ariely, D., Rolland, J. P., and Martin, K. E. 1996. Occlusion edge blur: a cue to relative visual depth. Journal of the Optical Society of America. A, Optics, image science, and vision 13, 4 (Apr.), 681--8.

[41]

Masia, B., Wetzstein, G., Didyk, P., and Gutierrez, D. 2013. A survey on computational displays: Pushing the boundaries of optics, computation, and perception. Computers and Graphics 37, 8, 1012--1038.

Digital Library

[42]

Mather, G., and Smith, D. R. R. 2002. Blur discrimination and its relation to blur-mediated depth perception. Perception 31, 10, 1211--1219.

[43]

Mathews, S., and Kruger, P. 1994. Spatiotemporal transfer function of human accommodation. Vision Research 34, 15.

[44]

Matusik, W., and Pfister, H. 2004. 3D TV: A scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. ACM Trans. Graph. 23, 3 (Aug.), 814--824.

Digital Library

[45]

Mendiburu, B. 2009. 3D movie making: Stereoscopic digital cinema from script to screen. Focal Press, Elsevier.

[46]

Navarro, R. 2009. The optical design of the human eye: a critical review. Journal of Optometry 2, 1, 3--18.

[47]

Owens, D. 1980. A comparison of accommodative responsiveness and contrast sensitivity for sinusoidal gratings. Vision Research 20, 2, 159--167.

[48]

Palmer, S. E., and Brooks, J. L. 2008. Edge-region grouping in figure-ground organization and depth perception. Journal of Experimental Psychology: Human Perception and Performance 34, 6 (Dec), 1353--1371.

[49]

Pamplona, V. F., Oliveira, M. M., Aliaga, D. G., and Raskar, R. 2012. Tailored displays to compensate for visual aberrations. ACM Trans. Graph. 31, 4 (July), 81:1--81:12.

Digital Library

[50]

Perlin, K., Paxia, S., and Kollin, J. S. 2000. An autostereoscopic display. In Proc. 27th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '00, 319--326.

Digital Library

[51]

Ravikumar, S., Akeley, K., and Banks, M. S. 2011. Creating effective focus cues in multi-plane 3D displays. Opt. Express 19, 21 (Oct), 20940--20952.

[52]

Ryan, L., MacKenzie, K., and Watt, S. 2012. Multiple-focal-planes 3D displays: A practical solution to the vergence-accommodation conflict? In 3D Imaging (IC3D), 2012 International Conference on, 1--6.

[53]

Shibata, T., Kim, J., Hoffman, D. M., and Banks, M. S. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. Journal of Vision 11, 8.

[54]

Spring, K., and Stiles, W. S. 1948. Variation of pupil size with change in the angle at which the light stimulus strikes the retina. British J. Ophthalmol. 32, 6, 340--346.

[55]

Sullivan, A. 2004. DepthCube solid-state 3D volumetric display. In Proc. SPIE, vol. 5291, 279--284.

[56]

Takaki, Y., Tanaka, K., and Nakamura, J. 2011. Super multi-view display with a lower resolution flat-panel display. Opt. Express 19, 5 (Feb), 4129--4139.

[57]

Takaki, Y. 2006. High-density directional display for generating natural three-dimensional images. Proc. IEEE 94, 3, 654--663.

[58]

van Ee, R., Banks, M. S., and T., B. B. 1999. An analysis of binocular slant contrast. Perception 28, 9, 1121--1145.

[59]

Watt, S. J., Akeley, K., Ernst, M. O., and Banks, M. S. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10.

[60]

Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R. 2011. Layered 3D: Tomographic image synthesis for attenuation-based light field and high dynamic range displays. ACM Trans. Graph. 30, 4.

Digital Library

[61]

Wetzstein, G., Lanman, D., Gutierrez, D., and Hirsch, M., 2012. Computational displays: Combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future. ACM SIGGRAPH Course Notes.

Digital Library

[62]

Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R. 2012. Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4, 1--11.

Digital Library

[63]

Zannoli, M., Albert, R. A., Bulbul, A., Narain, R., O'Brien, J. F., and Banks, M. S. 2014. Correct blur and accommodation information is a reliable cue to depth ordering. In Proc. Vision Sciences Society 14th Annual Meeting.

[64]

Zhu, M., and Chan, T. 2008. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. Tech. rep., University of California, Los Angeles.

[65]

Zwicker, M., Matusik, W., Durand, F., Pfister, H., and Forlines, C. 2006. Antialiasing for automultiscopic 3D displays. In ACM SIGGRAPH 2006 Sketches, SIGGRAPH '06.

Digital Library

Cited By

Zhao NXiao JWeng PZhang H(2024)Tomographic waveguide-based augmented reality displayOptics Express10.1364/OE.524983Online publication date: 30-Apr-2024
https://doi.org/10.1364/OE.524983
March JKrishnan AMantiuk RWatt S(2024)Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technologyJournal of Vision10.1167/jov.24.2.1324:2(13)Online publication date: 27-Feb-2024
https://doi.org/10.1167/jov.24.2.13
Ebner CPlopski ASchmalstieg DKalkofen D(2024)Gaze-Contingent Layered Optical See-Through Displays with a Confidence-Driven View VolumeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345620430:11(7203-7213)Online publication date: Nov-2024
https://doi.org/10.1109/TVCG.2024.3456204
Show More Cited By

Index Terms

Optimal presentation of imagery with focus cues on multi-plane displays
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Focal surface displays

Conventional binocular head-mounted displays (HMDs) vary the stimulus to vergence with the information in the picture, while the stimulus to accommodation remains fixed at the apparent distance of the display, as created by the viewing optics. Sustained ...
Fast gaze-contingent optimal decompositions for multifocal displays

As head-mounted displays (HMDs) commonly present a single, fixed-focus display plane, a conflict can be created between the vergence and accommodation responses of the viewer. Multifocal HMDs have long been investigated as a potential solution in which ...
Focus 3D: Compressive accommodation display

We present a glasses-free 3D display design with the potential to provide viewers with nearly correct accommodative depth cues, as well as motion parallax and binocular cues. Building on multilayer attenuator and directional backlight architectures, the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 34, Issue 4

August 2015

1307 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2809654

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2015

Published in TOG Volume 34, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

92
Total Citations
View Citations
1,050
Total Downloads

Downloads (Last 12 months)43
Downloads (Last 6 weeks)8

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao NXiao JWeng PZhang H(2024)Tomographic waveguide-based augmented reality displayOptics Express10.1364/OE.524983Online publication date: 30-Apr-2024
https://doi.org/10.1364/OE.524983
March JKrishnan AMantiuk RWatt S(2024)Impact of focus cue presentation on perceived realism of 3-D scene structure: Implications for scene perception and for display technologyJournal of Vision10.1167/jov.24.2.1324:2(13)Online publication date: 27-Feb-2024
https://doi.org/10.1167/jov.24.2.13
Ebner CPlopski ASchmalstieg DKalkofen D(2024)Gaze-Contingent Layered Optical See-Through Displays with a Confidence-Driven View VolumeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345620430:11(7203-7213)Online publication date: Nov-2024
https://doi.org/10.1109/TVCG.2024.3456204
Hu FYang PDou JDou L(2024)Detecting and tracking moving objects in defocus blur scenesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104259103(104259)Online publication date: Aug-2024
https://doi.org/10.1016/j.jvcir.2024.104259
Romeiro REisemann EMarroquim R(2024)Retinal pre-filtering for light field displaysComputers and Graphics10.1016/j.cag.2024.104033123:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.104033
Qin YChen WO'Toole MSankaranarayanan A(2023)Split-Lohmann Multifocal DisplaysACM Transactions on Graphics10.1145/359211042:4(1-18)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592110
Akşit KItoh Y(2023)HoloBeam: Paper-Thin Near-Eye Displays2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR55154.2023.00073(581-591)Online publication date: Mar-2023
https://doi.org/10.1109/VR55154.2023.00073
Ebner CMohr PLanglotz TPeng YSchmalstieg DWetzstein GKalkofen D(2023)Off-Axis Layered Displays: Hybrid Direct-View/Near-Eye Mixed Reality with Focus CuesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324707729:5(2816-2825)Online publication date: May-2023
https://doi.org/10.1109/TVCG.2023.3247077
Chao BLiu CChen H(2023)Time-Division Multiplexing Light Field Display With Learned Coded ApertureIEEE Transactions on Image Processing10.1109/TIP.2022.320321032(350-363)Online publication date: 2023
https://doi.org/10.1109/TIP.2022.3203210
Liu JZhong FMantel CForchhammer SMantiuk R(2023)Computational 3D displaysImmersive Video Technologies10.1016/B978-0-32-391755-1.00023-7(469-500)Online publication date: 2023
https://doi.org/10.1016/B978-0-32-391755-1.00023-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents