More Web Proxy on the site http://driver.im/

research-article

ProtoAR: Rapid Physical-Digital Prototyping of Mobile Augmented Reality Applications

Authors:

Michael Nebeling,

Janet Nebeling,

Rob RumbleAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 353, Pages 1 - 12

https://doi.org/10.1145/3173574.3173927

Published: 21 April 2018 Publication History

Abstract

The latest generations of smartphones with built-in AR capabilities enable a new class of mobile apps that merge digital and real-world content depending on a user's task, context, and preference. But even experienced mobile app designers face significant challenges: creating 2D/3D AR content remains difficult and time-consuming, and current mobile prototyping tools do not support AR views. There are separate tools for this; however, they require significant technical skill. This paper presents ProtoAR which supplements rapid physical prototyping using paper and Play-Doh with new mobile cross-device multi-layer authoring and interactive capture tools to generate mobile screens and AR overlays from paper sketches, and quasi-3D content from 360-degree captures of clay models. We describe how ProtoAR evolved over four design jams with students to enable interactive prototypes of mobile AR apps in less than 90 minutes, and discuss the advantages and insights ProtoAR can give designers.

Supplementary Material

suppl.mov (pn3176-file5.mp4)

Supplemental video

Download
12.87 MB

References

[1]

Michelle Annett, Tovi Grossman, Daniel J. Wigdor, and George W. Fitzmaurice. 2015. MoveableMaker: Facilitating the Design, Generation, and Assembly of Moveable Papercraft. In Proc. UIST. 565--574.

Digital Library

[2]

Ronald Azuma. 1997. A Survey of Augmented Reality. Presence 6, 4 (1997), 355--385.

Digital Library

[3]

Ronald Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre. 2001. Recent Advances in Augmented Reality. IEEE Computer Graphics and Applications 21, 6 (2001), 34--47.

Digital Library

[4]

Marion Buchenau and Jane Fulton Suri. 2000. Experience Prototyping. In Proc. DIS. 424--433.

Digital Library

[5]

Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan Kaufmann.

Digital Library

[6]

Pei-Yu (Peggy) Chi and Yang Li. 2015. Weave: Scripting Cross-Device Wearable Interaction. In Proc. CHI.

Digital Library

[7]

Alan Cooper, Robert Reimann, David Cronin, and Christopher Noessel. 2014. About face: the essentials of interaction design. John Wiley & Sons.

Digital Library

[8]

Steven Dow, Jaemin Lee, Christopher Oezbek, Blair MacIntyre, Jay David Bolter, and Maribeth Gandy. 2005a. Wizard of Oz interfaces for mixed reality applications. In Proc. CHI Extended Abstracts. 1339--1342.

Digital Library

[9]

Steven Dow, Blair MacIntyre, Jaemin Lee, Christopher Oezbek, Jay David Bolter, and Maribeth Gandy. 2005b. Wizard of Oz support throughout an iterative design process. IEEE Pervasive Computing 4, 4 (2005), 18--26.

Digital Library

[10]

Maribeth Gandy and Blair MacIntyre. 2014. Designer's augmented reality toolkit, ten years later: implications for new media authoring tools. In Proc. UIST.

Digital Library

[11]

Peter Hamilton and Daniel J. Wigdor. 2014. Conductor: Enabling and Understanding Cross-Device Interaction. In Proc. CHI.

Digital Library

[12]

Steven Houben and Nicolai Marquardt. 2015. WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. In Proc. CHI.

Digital Library

[13]

Andrew J. Hunsucker, Kelly McClinton, Jennifer Wang, and Erik Stolterman. 2017. Augmented Reality Prototyping For Interaction Design Students. In Proc. CHI Extended Abstracts. 1018--1023.

Digital Library

[14]

Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard A. Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew J. Davison, and Andrew W. Fitzgibbon. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. UIST. 559--568.

Digital Library

[15]

Bret Jackson and Daniel F. Keefe. 2016. Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. TVCG 22, 4 (2016).

Digital Library

[16]

Konstantin Klamka and Raimund Dachselt. 2017. IllumiPaper: Illuminated Interactive Paper. In Proc. CHI. 5605--5618.

Digital Library

[17]

Scott R. Klemmer, Jack Li, James Lin, and James A. Landay. 2004. Papier-Mache: toolkit support for tangible input. In Proc. CHI. 399--406.

Digital Library

[18]

Scott R. Klemmer, Mark W. Newman, Ryan Farrell, Mark Bilezikjian, and James A. Landay. 2001. The designers' outpost: a tangible interface for collaborative web site. In Proc. UIST. 1--10.

Digital Library

[19]

Boriana Koleva, Ian Taylor, Steve Benford, Mike Fraser, Chris Greenhalgh, Holger Schn¨ adelbach, Dirk vom Lehn, Christian Heath, Ju Row-Farr, and Matt Adams. 2001. Orchestrating a mixed reality performance. In Proc. CHI. 38--45.

Digital Library

[20]

James A. Landay and Brad A. Myers. 1995. Interactive Sketching for the Early Stages of User Interface Design. In Proc. CHI. 43--50.

Digital Library

[21]

Chunyuan Liao, Franc ¸ois Guimbreti` ere, Ken Hinckley, and James D. Hollan. 2008. Papiercraft: A gesture-based command system for interactive paper. ACM Trans. Comput.-Hum. Interact. 14, 4 (2008), 18:1--18:27.

Digital Library

[22]

James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: finding a tighter fit between tools and practice for Web site design. In Proc. CHI. 510--517.

Digital Library

[23]

Hao L¨ u and Yang Li. 2012. Gesture Coder: A Tool for Programming Multi-touch Gestures By Demonstration. In Proc. CHI.

Digital Library

[24]

Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proc. UIST.

Digital Library

[25]

Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321--1329.

[26]

Bill Moggridge and Bill Atkinson. 2007. Designing interactions. Vol. 17. MIT press Cambridge, MA.

Digital Library

[27]

Michael Nebeling. 2017. XDBrowser 2.0: Semi-Automatic Generation of Cross-Device Interfaces. In Proc. CHI.

Digital Library

[28]

Michael Nebeling and Anind K. Dey. 2016. XDBrowser: User-Defined Cross-Device Web Page Designs. In Proc. CHI.

Digital Library

[29]

Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira C. Norrie. 2014. Interactive Development of Cross-Device User Interfaces. In Proc. CHI.

Digital Library

[30]

Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic Books (AZ).

Digital Library

[31]

Peter Ondruska, Pushmeet Kohli, and Shahram Izadi. 2015. MobileFusion: Real-Time Volumetric Surface Reconstruction and Dense Tracking on Mobile Phones. TVCG 21, 11 (2015).

Digital Library

[32]

Thammathip Piumsomboon, Adrian J. Clark, Mark Billinghurst, and Andy Cockburn. 2013. User-Defined Gestures for Augmented Reality. In Proc. INTERACT. 282--299.

Digital Library

[33]

Marc Rettig. 1994. Prototyping for Tiny Fingers. Commun. ACM 37, 4 (1994), 21--27.

Digital Library

[34]

Yvonne Rogers, Helen Sharp, and Jenny Preece. 2011. Interaction design: beyond human-computer interaction. John Wiley & Sons.

Digital Library

[35]

Carolyn Snyder. 2003. Paper prototyping: The fast and easy way to design and refine user interfaces. Morgan Kaufmann.

Digital Library

[36]

Jie Song, G´ abor S¨ or¨ os, Fabrizio Pece, Sean Ryan Fanello, Shahram Izadi, Cem Keskin, and Otmar Hilliges. 2014. In-air gestures around unmodified mobile devices. In Proc. UIST. 319--329.

Digital Library

[37]

Eugene M. Taranta II, Amirreza Samiei, Mehran Maghoumi, Pooya Khaloo, Corey R. Pittman, and Joseph J. LaViola Jr. 2017. Jackknife: A Reliable Recognizer with Few Samples and Many Modalities. In Proc. CHI. 5850--5861.

Digital Library

[38]

Daniel Vogel and Ravin Balakrishnan. 2004. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In Proc. UIST.

Digital Library

[39]

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li. 2007. Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proc. UIST.

Digital Library

[40]

Feng Zhou, Henry Been-Lirn Duh, and Mark Billinghurst. 2008. Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR. In Proc. ISMAR. 193--202.

Digital Library

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Lee LLin Z(2024)Danger, Nuisance, Disregard: Analyzing User-Generated Videos for Augmented Reality Gameplay on Hand-held DevicesProceedings of the ACM on Human-Computer Interaction10.1145/36770638:CHI PLAY(1-33)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677063
Rajaram S(2024)Enabling Safer Augmented Reality Experiences: Usable Privacy Interventions for AR Creators and End-UsersAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686708(1-8)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686708
Show More Cited By

Index Terms

ProtoAR: Rapid Physical-Digital Prototyping of Mobile Augmented Reality Applications
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

360proto: Making Interactive Virtual Reality & Augmented Reality Prototypes from Paper
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

We explore 360 paper prototyping to rapidly create AR/VR prototypes from paper and bring them to life on AR/VR devices. Our approach is based on a set of emerging paper prototyping templates specifically for AR/VR. These templates resemble the key ...
A Survey on Haptic Technologies for Mobile Augmented Reality
Augmented reality (AR) applications have gained much research and industry attention. Moreover, the mobile counterpart—mobile augmented reality (MAR) is one of the most explosive growth areas for AR applications in the mobile environment (e.g., ...
Mobile augmented reality learning tool to simulate experts' perspectives in the field
UbiComp '12: Proceedings of the 2012 ACM Conference on Ubiquitous Computing

Mobile Augmented Reality (MAR)---techniques to dynamically overlay digital information in the user's view through mobile devices---has the potential to uniquely contribute to people learning about objects or environments in situ. Using interactive ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
2,013
Total Downloads

Downloads (Last 12 months)151
Downloads (Last 6 weeks)39

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Lee LLin Z(2024)Danger, Nuisance, Disregard: Analyzing User-Generated Videos for Augmented Reality Gameplay on Hand-held DevicesProceedings of the ACM on Human-Computer Interaction10.1145/36770638:CHI PLAY(1-33)Online publication date: 15-Oct-2024
https://dl.acm.org/doi/10.1145/3677063
Rajaram S(2024)Enabling Safer Augmented Reality Experiences: Usable Privacy Interventions for AR Creators and End-UsersAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686708(1-8)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3672539.3686708
Zhang LPan JGettig JOney SGuo A(2024)VRCopilot: Authoring 3D Layouts with Generative AI Models in VRProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676451(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676451
Nguyen QWindisch AKriglstein S(2024)Between Two Worlds: Analysing the Effects of Immersive and Non-Immersive Prototyping for Participatory DesignProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660715(2198-2212)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660715
Jin LFagan D(2024)MRPaperPro: Mixed Reality Interdisciplinary Design Prototyping TechniqueProceedings of the 2024 ACM International Conference on Interactive Media Experiences10.1145/3639701.3656310(268-275)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3639701.3656310
Huang XXiao R(2024)SurfShareProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314187:4(1-24)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631418
Chen ZJohnson TKnowles ALi AYi SZhuang YByrne DEl-Zanfaly DCiolfi LHogan TDöring TJenkins Tvan Dijk JHuron SLi ZCoyle DSigner B(2024)Augmenting Embodied Learning in Welding Training: The Co-Design of an XR- and tinyML-Enabled Welding System for Creative Arts and Manufacturing TrainingProceedings of the Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3623509.3633398(1-14)Online publication date: 11-Feb-2024
https://dl.acm.org/doi/10.1145/3623509.3633398
Jo HSuzuki RKim Y(2024)CollageVis: Rapid Previsualization Tool for Indie Filmmaking using Video CollagesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642575(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642575
Cho HYan YTodi KParent MSmith MJonker TBenko HLindlbauer D(2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642394
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents