More Web Proxy on the site http://driver.im/

research-article

SnapToReality: Aligning Augmented Reality to the Real World

Authors:

Benjamin Nuernberger,

Andrew D. WilsonAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 1233 - 1244

https://doi.org/10.1145/2858036.2858250

Published: 07 May 2016 Publication History

Abstract

Augmented Reality (AR) applications may require the precise alignment of virtual objects to the real world. We propose automatic alignment of virtual objects to physical constraints calculated from the real world in real time ("snapping to reality"). We demonstrate SnapToReality alignment techniques that allow users to position, rotate, and scale virtual content to dynamic, real world scenes. Our proof-of-concept prototype extracts 3D edge and planar surface constraints. We furthermore discuss the unique design challenges of snapping in AR, including the user's limited field of view, noise in constraint extraction, issues with changing the view in AR, visualizing constraints, and more. We also report the results of a user study evaluating SnapToReality, confirming that aligning objects to the real world is significantly faster when assisted by snapping to dynamically extracted constraints. Perhaps more importantly, we also found that snapping in AR enables a fresh and expressive form of AR content creation.

Supplementary Material

ZIP File (pn1170-file4.zip)

pn1170-file4.zip

Download
.54 KB

suppl.mov (pn1170.mp4)

Supplemental video

Download
71.27 MB

MP4 File (p1233-nuernberger.mp4)

Download
224.03 MB

References

[1]

Patrick Baudisch, Edward Cutrell, Ken Hinckley, and Adam Eversole. 2005. Snap-and-go: helping users align objects without the modality of traditional snapping. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '05), 301--310. http://doi.org/10.1145/1054972.1055014

Digital Library

[2]

Blaine Bell, Steven Feiner, and Tobias Höllerer. 2001. View management for virtual and augmented reality. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST '01), 101--110. http://doi.org/10.1145/502348.502363

Digital Library

[3]

Eric A. Bier and Maureen C. Stone. 1986. Snapdragging. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '86), 233--240. http://dx.doi.org/10.1145/15922.15912

Digital Library

[4]

Eric A. Bier. 1990. Snap-dragging in three dimensions. In Proceedings of the 1990 Symposium on Interactive 3D Graphics (I3D '90), 193--204. http://doi.org/10.1145/91385.91446

Digital Library

[5]

Dorit Borrmann, Jan Elseberg, Kai Lingemann, and Andreas Nüchter. 2011. The 3D Hough Transform for plane detection in point clouds: A review and a new accumulator design. 3D Research. 2, 2, Article 32 (June 2011): 1--13. http://dx.doi.org/10.1007/3DRes.02(2011)3

Digital Library

[6]

John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). 8, 6 (Nov. 1986): 679698. http://doi.org/10.1109/TPAMI.1986.4767851

Digital Library

[7]

Changhyun Choi, Alexander J. B. Trevor, and Henrik I. Christensen. 2013. RGB-D Edge Detection and Edgebased Registration. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '13), 1568--1575. http://doi.org/10.1109/IROS.2013.6696558

[8]

Richard O. Duda and Peter E. Hart. 1972. Use of the Hough transformation to detect lines and curves in pictures. Communications of the ACM. 15, 1 (Jan. 1972): 11--15. http://doi.org/10.1145/361237.361242

Digital Library

[9]

Leandro A. F. Fernandes and Manuel M. Oliveira. 2008. Real-time line detection through an improved Hough transform voting scheme. Pattern Recognition. 41, 1 (Jan. 2008): 299--314. http://dx.doi.org/10.1016/j.patcog.2007.04.003

Digital Library

[10]

Jennifer Fernquist, Garth Shoemaker, and Kellogg S. Booth. 2011. "Oh Snap"--Helping Users Align Digital Objects on Touch Interfaces. In Proceedings of the 13th IFIP TC 13 International Conference on HumanComputer Interaction (INTERACT 2011), 338--355. http://dx.doi.org/10.1007/978--3--642--23765--2_24

Digital Library

[11]

Martin A. Fischler and Robert C. Bolles. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM. 24, 6 (June 1981): 381--395. http://doi.org/10.1145/358669.358692

Digital Library

[12]

Carlos H. Q. Forster and Clésio L. Tozzi. 2001. An architecture based on constraints for augmented shared workspaces. In Proceedings of the 14th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI '01), 328--335. http://doi.org/10.1109/SIBGRAPI.2001.963073

Digital Library

[13]

Mathias Frisch, Ricardo Langner, and Raimund Dachselt. 2011. Neat: a set of flexible tools and gestures for layout tasks on interactive displays. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '11), 1--10. http://doi.org/10.1145/2076354.2076356

Digital Library

[14]

Ran Gal, Lior Shapira, Eyal Ofek, and Pushmeet Kohli. 2014. FLARE: Fast layout for augmented reality applications. In Proceedings of the 13th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '14), 207--212. http://doi.org/10.1109/ISMAR.2014.6948429

[15]

Natasha Gelfand and Leonidas J. Guibas. 2004. Shape segmentation using local slippage analysis. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP '04), 214--223. http://doi.org/10.1145/1057432.1057461

Digital Library

[16]

Michael Gleicher. 1995. Image snapping. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '95), 183--190. http://doi.org/10.1145/218380.218441

Digital Library

[17]

Seongkook Heo, Yong-Ki Lee, Jiho Yeom, and Geehyuk Lee. 2012. Design of a shape dependent snapping algorithm. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 2207--2212. http://doi.org/10.1145/2212776.2223777

Digital Library

[18]

Paul V. C. Hough. 1962. Method and means for recognizing complex patterns. (Dec. 1962). US Patent 3069654 A, Filed Mar. 25, 1960, Issued Dec. 18, 1962.

[19]

Klaas Klasing, Daniel Althoff, Dirk Wollherr, and Martin Buss. 2009. Comparison of surface normal estimation methods for range sensing applications. In Proceedings of the 26th IEEE International Conference on Robotics and Automation (ICRA '09), 3206--3211. http://doi.org/10.1109/ROBOT.2009.5152493

Digital Library

[20]

Gun A. Lee, Ungyeon Yang, Yongwan Kim, Dongsik Jo, and Ki-Hong Kim. Snap-to-feature interface for annotation in mobile augmented reality. In Augmented Reality Super Models Workshop at the 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '10), Retrieved September 14, 2015 from http://www.icg.tugraz.at/Members/arth/arsupermodels/ 04_lee.pdf

[21]

Gun A. Lee and Mark Billinghurst. 2012. Assistive techniques for precise touch interaction in handheld augmented reality environments. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry (VRCAI '12), 279--285. http://doi.org/10.1145/2407516.2407582

Digital Library

[22]

Paul Merrell, Eric Schkufza, Zeyang Li, Maneesh Agrawala, and Vladlen Koltun. 2011. Interactive furniture layout using interior design guidelines. ACM Transactions on Graphics (TOG). 30, 4, Article 87 (July 2011): 1--10. http://dx.doi.org/10.1145/2010324.1964982

Digital Library

[23]

Eric N. Mortensen and William A. Barrett. 1998. Interactive segmentation with intelligent scissors. Graphical Models and Image Processing. 60, 5 (Sept. 1998): 349--384. http://dx.doi.org/10.1006/gmip.1998.0480

Digital Library

[24]

Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. 2015. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR '15), 343--352. http://dx.doi.org/10.1109/CVPR.2015.7298631

[25]

Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J. Davison, Pushmeet Kohli, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time dense surface mapping and tracking. In Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR '11), 127--136. http://doi.org/10.1109/ISMAR.2011.6092378

Digital Library

[26]

Rui Nóbrega and Nuno Correia. 2012. Magnetic augmented reality. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI '12), 332--335. http://doi.org/10.1145/2254556.2254620

Digital Library

[27]

Ji-Young Oh, Wolfgang Stuerzlinger, and John Danahy. 2006. SESAME: towards better 3D conceptual design systems. In Proceedings of the 6th Conference on Designing Interactive Systems (DIS '06), 80--89. http://doi.org/10.1145/1142405.1142419

Digital Library

[28]

Nathan Silberman, Lior Shapira, Ran Gal, and Pushmeet Kohli. 2014. A contour completion model for augmenting surface reconstructions. In Proceedings of the 14th European Conference on Computer Vision (ECCV '14), 488--503. http://dx.doi.org/10.1007/978--3319--10578--9_32

[29]

Mihran Tuceryan and Nassir Navab. 2000. Single point active alignment method (SPAAM) for optical seethrough HMD calibration for AR. In Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR '00), 149--158. http://doi.org/10.1109/ISAR.2000.880938

[30]

Ken Xu, James Stewart, and Eugene Fiume. 2002. Constraint-based Automatic Placement for Scene Composition. In Proceedings of the Graphics Interface Conference (GI '02), 25--34. Retrieved September 14, 2015 from http://graphicsinterface.org/wpcontent/uploads/gi2002--4.pdf

[31]

Pengfei Xu, Hongbo Fu, Chiew-Lan Tai, and Takeo Igarashi. 2015. GACA: Group-Aware Command-based Arrangement of Graphic Elements. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2787--2795. http://doi.org/10.1145/2702123.2702198

Digital Library

Cited By

Nguyen LHtet HLee YPark M(2024)Augmented Reality Framework for Retrieving Information of Moving Objects on Construction SitesBuildings10.3390/buildings1407208914:7(2089)Online publication date: 8-Jul-2024
https://doi.org/10.3390/buildings14072089
He FHu XQian XZhu ZRamani K(2024)AdapTUI: Adaptation of Geometric-Feature-Based Tangible User Interfaces in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981278:ISS(44-69)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698127
Chidambaram SWilliams ABai MVirk SHaffner PLease MLi E(2024)Annorama: Enabling Immersive At-Desk Annotation Experiences in Virtual Reality with 3D Point Cloud DioramasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682081(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682081
Show More Cited By

Index Terms

SnapToReality: Aligning Augmented Reality to the Real World
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Recommendations

Investigating the balance between virtuality and reality in mobile mixed reality UI design: user perception of an augmented city
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational

Examples of mixed reality mobile applications and research combining virtual and real world data in the same view have emerged during recent years. However, currently there is little knowledge of users' perceptions comparing the role of virtual and real ...
An Interaction Design Model for Information Visualization in Immersive Augmented Reality platform
MoMM2019: Proceedings of the 17th International Conference on Advances in Mobile Computing & Multimedia

Recent advances in immersive technologies such as Augmented Reality (AR) provide new opportunities to explore, analyze and present data through Immersive Analytics (IA). This study presents a thorough overview of current research in immersive AR and ...
A gesture control system for intuitive 3D interaction with virtual objects
VRCAI 08

We present a system for interacting with 3D objects in a 3D virtual environment. Using the notion that a typical head-mounted display (HMD) does not cover the user's entire face, we use a fiducial marker placed on the HMD to locate the user's exposed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

96
Total Citations
View Citations
2,209
Total Downloads

Downloads (Last 12 months)195
Downloads (Last 6 weeks)26

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nguyen LHtet HLee YPark M(2024)Augmented Reality Framework for Retrieving Information of Moving Objects on Construction SitesBuildings10.3390/buildings1407208914:7(2089)Online publication date: 8-Jul-2024
https://doi.org/10.3390/buildings14072089
He FHu XQian XZhu ZRamani K(2024)AdapTUI: Adaptation of Geometric-Feature-Based Tangible User Interfaces in Augmented RealityProceedings of the ACM on Human-Computer Interaction10.1145/36981278:ISS(44-69)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3698127
Chidambaram SWilliams ABai MVirk SHaffner PLease MLi E(2024)Annorama: Enabling Immersive At-Desk Annotation Experiences in Virtual Reality with 3D Point Cloud DioramasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682081(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682081
Chang YWang YChang CTan WHsu YChen YChen M(2024)Experience from Designing Augmented Reality Browsing Interfaces for Real-world Walking ScenariosProceedings of the ACM on Human-Computer Interaction10.1145/36765008:MHCI(1-26)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676500
Lipkowitz G(2024)Palimpsest: a spatial user interface toolkit for cohering tracked physical entities and interactive 3D contentAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686780(1-4)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686780
Leisi CGötz U(2024)Virtual Real Spaces: In-Home Gaming Experiences Bridging Reality and VirtualityCompanion Proceedings of the 2024 Annual Symposium on Computer-Human Interaction in Play10.1145/3665463.3678812(157-162)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3665463.3678812
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Williams ABai MBuck JMckinney TRechkemmer AKalyanaraman KLease MHaffner PZhou XLi E(2024)Snapper: Accelerating Bounding Box Annotation in Object Detection Tasks with Find-and-Snap ToolingProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645162(471-488)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645162
Luo W(2024)Exploring Spatial Organization Strategies for Virtual Content in Mixed Reality EnvironmentsExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638181(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638181
Cho HYan YTodi KParent MSmith MJonker TBenko HLindlbauer D(2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642394
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents