research-article

Public Access

Scene-adaptive high dynamic range display for low latency augmented reality

Authors:

Henry FuchsAuthors Info & Claims

I3D '17: Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games

Article No.: 15, Pages 1 - 7

https://doi.org/10.1145/3023368.3023379

Published: 25 February 2017 Publication History

PDF eReader

Abstract

For generalized augmented reality to be feasible, the augmenting elements must be visible in varied environments and under rapidly changing, high dynamic range lighting, from bright sunlight to deep shadows. We present a high dynamic range, optical see-through, augmented reality display that dynamically adjusts the brightness of the virtual imagery to match the current brightness of the real scene. Critical components include the spatial brightness sensor array and the positional brightness image intensity matcher. The color, scene-adaptive HDR display system is based on a high-rate (15 kHz) DMD projector using a high-speed RGB LED illuminator, each color with independent 16 bit intensity control for each binary DMD frame. The critical input to the intensity matching algorithm is the output of an array of high sensitivity light sensors. This paper discusses the implementation of the system and reports performance via still and video demonstrations under a variety of lighting conditions.

Supplementary Material

ZIP File (a15-lincoln.zip)

Supplemental material.

Download
103.04 MB

References

[1]

Greer, T., Spjut, J., Luebke, D., and Whitted, T. 2016. 8--3: Hybrid modulation for near zero display latency. SID Symposium Digest of Technical Papers 47, 1, 76--78.

Crossref

Google Scholar

[2]

Horowitz, P., and Hill, W. 2015. The Art of Electronics, 3rd ed. Cambridge University Press, April.

Digital Library

Google Scholar

[3]

Itoh, Y., Orlosky, J., Huber, M., Kiyokawa, K., and Klinker, G. 2016. OST Rift: Temporally consistent augmented reality with a consumer optical see-through head-mounted display. In 2016 IEEE Virtual Reality (VR), 189--190.

Google Scholar

[4]

Lincoln, P., Blate, A., Singh, M., Whitted, T., State, A., Lastra, A., and Fuchs, H. 2016. From motion to photons in 80 microseconds: Towards minimal latency for virtual and augmented reality. IEEE Transactions on Visualization and Computer Graphics 22, 4 (April), 1367--1376.

Digital Library

Google Scholar

[5]

Mann, S., Lo, R. C. H., Ovtcharov, K., Gu, S., Dai, D., Ngan, C., and Ai, T. 2012. Realtime HDR (high dynamic range) video for eyetap wearable computers, FPGA-based seeing aids, and glasseyes (EyeTaps). In 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE), 1--6.

Google Scholar

[6]

Pasman, W., van der Schaaf, A., Lagendijk, R., and Jansen, F. 1999. Low latency rendering for mobile augmented reality. In Proceedings of the ASCI, vol. 99, 15--17.

Google Scholar

[7]

Pavlovych, A., and Stuerzlinger, W. 2005. A high-dynamic range projection system. Proc. SPIE 5969, 59692O--59692O-8.

Google Scholar

[8]

Regan, M., and Pose, R. 1994. Priority rendering with a virtual reality address recalculation pipeline. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '94, 155--162.

Digital Library

Google Scholar

[9]

Seetzen, H., Whitehead, L. A., and Ward, G. 2003. 54.2: A high dynamic range display using low and high resolution modulators. SID Symposium Digest of Technical Papers 34, 1, 1450--1453.

Crossref

Google Scholar

[10]

Wetzstein, G., Heidrich, W., and Luebke, D. 2010. Optical Image Processing Using Light Modulation Displays. Computer Graphics Forum 29, 1934--1944.

Crossref

Google Scholar

[11]

Zheng, F., Whitted, T., Lastra, A., Lincoln, P., State, A., Maimone, A., and Fuchs, H. 2014. Minimizing latency for augmented reality displays: Frames considered harmful. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on, 195--200.

Google Scholar

Cited By

View all

Khan TZhu TDownes TCheng LKass NAndrews EBiehl J(2023)Understanding Effects of Visual Feedback Delay in AR on Fine Motor Surgical TasksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021429:11(4697-4707)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3320214
Itoh YLanglotz TSutton JPlopski A(2021)Towards Indistinguishable Augmented RealityACM Computing Surveys10.1145/345315754:6(1-36)Online publication date: 13-Jul-2021
https://dl.acm.org/doi/10.1145/3453157
Fan W(2021)A Fast and Realistic Bloom Rendering Method for Large Scale 3D Scene2021 11th International Conference on Information Science and Technology (ICIST)10.1109/ICIST52614.2021.9440634(178-182)Online publication date: 21-May-2021
https://doi.org/10.1109/ICIST52614.2021.9440634
Show More Cited By

Index Terms

Scene-adaptive high dynamic range display for low latency augmented reality
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces

Recommendations

Experiencing Social Augmented Reality in Public Spaces
UbiComp/ISWC '21 Adjunct: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers

We present various views on the future use of augmented reality in public spaces. The views address enhanced walking, social activity, appropriation of public spaces, and futuristic social aspects of future outdoor augmented reality. Although we often ...
An optical see-through augmented reality system for the treatment of phobia to small animals
ICVR'07: Proceedings of the 2nd international conference on Virtual reality

This paper presents an optical see-through (OST) Augmented Reality (AR) system for the treatment of phobia to small animals. Our group has already presented two AR systems for this treatment. The first system was a video see-through (VST) AR system. The ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

I3D '17: Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games

February 2017

161 pages

ISBN:9781450348867

DOI:10.1145/3023368

Editor:
Stephen N. Spencer
University of Washington
,
General Chairs:
Adam Bargteil
University of Maryland, Baltimore County
,
Kenny Mitchell
Disney Research and Edinburgh Napier University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation
BeingThere / BeingTogether Centre

Conference

I3D '17

Sponsor:

SIGGRAPH

I3D '17: Symposium on Interactive 3D Graphics and Games

February 25 - 27, 2017

California, San Francisco

Acceptance Rates

I3D '17 Paper Acceptance Rate 16 of 45 submissions, 36%;

Overall Acceptance Rate 148 of 485 submissions, 31%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
950
Total Downloads

Downloads (Last 12 months)124
Downloads (Last 6 weeks)15

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Khan TZhu TDownes TCheng LKass NAndrews EBiehl J(2023)Understanding Effects of Visual Feedback Delay in AR on Fine Motor Surgical TasksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332021429:11(4697-4707)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3320214
Itoh YLanglotz TSutton JPlopski A(2021)Towards Indistinguishable Augmented RealityACM Computing Surveys10.1145/345315754:6(1-36)Online publication date: 13-Jul-2021
https://dl.acm.org/doi/10.1145/3453157
Fan W(2021)A Fast and Realistic Bloom Rendering Method for Large Scale 3D Scene2021 11th International Conference on Information Science and Technology (ICIST)10.1109/ICIST52614.2021.9440634(178-182)Online publication date: 21-May-2021
https://doi.org/10.1109/ICIST52614.2021.9440634
Xiong JHsiang EHe ZZhan TWu S(2021)Augmented reality and virtual reality displays: emerging technologies and future perspectivesLight: Science & Applications10.1038/s41377-021-00658-810:1Online publication date: 25-Oct-2021
https://doi.org/10.1038/s41377-021-00658-8
Blate AWhitton MSingh MWelch GState AWhitted TFuchs H(2019)Implementation and Evaluation of a 50 kHz, $28\mu\mathrm{s}$ Motion-to-Pose Latency Head Tracking InstrumentIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2019.289923325:5(1970-1980)Online publication date: May-2019
https://doi.org/10.1109/TVCG.2019.2899233
Chang JKumar BSankaranarayanan A(2018)Towards multifocal displays with dense focal stacksACM Transactions on Graphics10.1145/3272127.327501537:6(1-13)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275015
Rathinavel KWang HBlate AFuchs H(2018)An Extended Depth-at-Field Volumetric Near-Eye Augmented Reality DisplayIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.286857024:11(2857-2866)Online publication date: Nov-2018
https://doi.org/10.1109/TVCG.2018.2868570
Akşit KLopes WKim JShirley PLuebke D(2017)Near-eye varifocal augmented reality display using see-through screensACM Transactions on Graphics10.1145/3130800.313089236:6(1-13)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130892
Mori SIkeda SSandor CPlopski A(2017)[POSTER] BrightView: Increasing Perceived Brightness in Optical See-Through Head-Mounted Displays2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct)10.1109/ISMAR-Adjunct.2017.66(202-203)Online publication date: Oct-2017
https://doi.org/10.1109/ISMAR-Adjunct.2017.66

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Experiencing Social Augmented Reality in Public Spaces

An optical see-through augmented reality system for the treatment of phobia to small animals

Extending Virtual Reality Display Wall Environments Using Augmented Reality