More Web Proxy on the site http://driver.im/

research-article

Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality

Authors:

Philipp Wacker,

Jan BorchersAuthors Info & Claims

SUI '18: Proceedings of the 2018 ACM Symposium on Spatial User Interaction

Pages 25 - 35

https://doi.org/10.1145/3267782.3267788

Published: 13 October 2018 Publication History

Abstract

Besides sketching in mid-air, Augmented Reality (AR) lets users sketch 3D designs directly attached to existing physical objects. These objects provide natural haptic feedback whenever the pen touches them, and, unlike in VR, there is no need to digitize the physical object first. Especially in Personal Fabrication, this lets non-professional designers quickly create simple 3D models that fit existing physical objects, such as a lampshade for a lamp socket. We categorize guidance types of real objects into flat, concave, and convex surfaces, edges, and surface markings. We studied how accurately these guides let users draw 3D shapes attached to physical vs. virtual objects in AR. Results show that tracing physical objects is 48% more accurate, and can be performed in a similar time compared to virtual objects. Guides on physical objects further improve accuracy especially in the vertical direction. Our findings provide initial metrics when designing AR sketching systems.

Supplementary Material

MP4 File (p25-wacker.mp4)

Download
314.81 MB

References

[1]

Rahul Arora, Rubaiat Habib Kazi, Tovi Grossman, George Fitzmaurice, and Karan Singh. 2018. SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ. In Proc CHI '18. ACM, 185:1--185:15.

Digital Library

[2]

Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. In Proc CHI '17. ACM, 5643--5654.

Digital Library

[3]

Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. In Proceedings of the 21st annual ACM symposium on User interface software and technology - UIST '08. ACM Press.

Digital Library

[4]

Ravin Balakrishnan, George Fitzmaurice, Gordon Kurtenbach, and Karan Singh. 1999. Exploring interactive curve and surface manipulation using a bend and twist sensitive input strip. In Proceedings of the 1999 symposium on Interactive 3D graphics - SI3D '99. ACM Press.

Digital Library

[5]

Frederick P. Brooks. 1999. What's real about virtual reality? IEEE Computer Graphics and Applications 19, 6 (1999), 16--27.

Digital Library

[6]

Xiang Cao and Shumin Zhai. 2007. Modeling human performance of pen stroke gestures. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '07. ACM Press.

Digital Library

[7]

Bruno R. De Araùjo, Géry Casiez, and Joaquim A. Jorge. 2012. Mockup Builder: Direct 3D Modeling on and Above the Surface in a Continuous Interaction Space. In Proceedings of Graphics Interface 2012 (GI '12). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 173--180. http://dl.acm.org/citation.cfm?id=2305276.2305305

Digital Library

[8]

Jennifer Fernquist, Tovi Grossman, and George Fitzmaurice. 2011. Sketch-sketch revolution: an engaging tutorial system for guided sketching and application learning. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST '11. ACM Press.

Digital Library

[9]

Matthew Flagg and James M. Rehg. 2006. Projector-guided painting. In Proceedings of the 19th annual ACM symposium on User interface software and technology - UIST '06. ACM Press.

Digital Library

[10]

Timo Fleisch, Gino Brunetti, Pedro Santos, and André Stork. {n. d.}. Stroke-input methods for immersive styling environments. In Proceedings Shape Modeling Applications, 2004. IEEE.

Digital Library

[11]

Wei Gai, Cheng Lin, Chenglei Yang, Yulong Bian, Chia Shen, Xiangxu Meng, Lu Wang, Juan Liu, Mingda Dong, and Chengjie Niu. 2017. Supporting Easy Physical-to-Virtual Creation of Mobile VR Maze Games. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press.

Digital Library

[12]

Cindy Grimm and Pushkar Joshi. 2012. Just DrawIt: A 3D Sketching System. In Proceedings of the International Symposium on Sketch-Based Interfaces and Modeling (SBIM '12). Eurographics Association, Goslar Germany, Germany, 121--130. http://dl.acm.org/citation.cfm?id=2331067.2331084

Digital Library

[13]

Tovi Grossman, Ravin Balakrishnan, Gordon Kurtenbach, George Fitzmaurice, Azam Khan, and Bill Buxton. 2002. Creating principal 3D curves with digital tape drawing. In Proceedings of the SIGCHI conference on Human factors in computing systems Changing our world, changing ourselves - CHI '02. ACM Press.

Digital Library

[14]

Sean Gustafson, Daniel Bierwirth, and Patrick Baudisch. 2010. Imaginary Interfaces: Spatial Interaction with Empty Hands and Without Visual Feedback. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). 3--12.

Digital Library

[15]

Emmanuel Iarussi, Adrien Bousseau, and Theophanis Tsandilas. 2013. The drawing assistant: automated drawing guidance and feedback from photographs. In Proceedings of the 26th annual ACM symposium on User interface software and technology - UIST '13. ACM Press.

Digital Library

[16]

Takeo Igarashi, Satoshi Matsuoka, and Hidehiko Tanaka. 1999. Teddy: a sketching interface for 3D freeform design. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques - SIGGRAPH '99. ACM Press.

Digital Library

[17]

IKEA. 2016. IKEA Launches Pilot Virtual Reality (VR) Kitchen Experience for HTC Vive on Steam. Blog. Retrieved August 24, 2017 from http://www.ikea.com/us/en/about_ikea/newsitem/040516_Virtual-Reality.

[18]

Brent Edward Insko. 2001. Passive haptics significantly enhances virtual environments. Ph.D. Dissertation. University of North Carolina at Chapel Hill.

[19]

Bred Jackson and Daniel F. Keefe. 2011. Sketching Over Props: Understanding and Interpreting 3D Sketch Input Relative to Rapid Prototype Props. In IUI '11 Sketch Recognition Workshop.

[20]

Bret Jackson and Daniel F. Keefe. 2016. Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR. IEEE Transactions on Visualization and Computer Graphics 22, 4 (April 2016), 1442--1451.

Digital Library

[21]

J. Adam Jones, J. Edward Swan, II, Gurjot Singh, Eric Kolstad, and Stephen R. Ellis. 2008. The Effects of Virtual Reality, Augmented Reality, and Motion Parallax on Egocentric Depth Perception. In Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization (APGV '08). ACM, New York, NY, USA, 9--14.

Digital Library

[22]

Daniel F. Keefe, Robert C. Zeleznik, and David H. Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE Transactions on Visualization and Computer Graphics 13, 5 (Sept. 2007), 1067--1081.

Digital Library

[23]

Krzysztof Jakub Kruszyński and Robert van Liere. 2009. Tangible props for scientific visualization: concept, requirements, application. Virtual Reality 13, 4 (July 2009), 235--244.

Digital Library

[24]

Manfred Lau, Greg Saul, Jun Mitani, and Takeo Igarashi. 2010. Modeling-in-context: user design of complementary objects with a single photo. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium. Eurographics Association, 17--24.

Digital Library

[25]

Jérémy Laviole and Martin Hachet. 2012. PapARt: Interactive 3D graphics and multi-touch augmented paper for artistic creation. In 2012 IEEE Symposium on 3D User Interfaces (3DUI). IEEE, 3--6.

[26]

Pedro Lopes, Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Providing Haptics to Walls & Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press.

Digital Library

[27]

William A. McNeely. {n. d.}. Robotic graphics: a new approach to force feedback for virtual reality. In Proceedings of IEEE Virtual Reality Annual International Symposium. IEEE.

Digital Library

[28]

Paul Milgram and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems E77-D (1994). http://vered. rose.utoronto.ca/people/paul_dir/IEICE94/ieice.html

[29]

Donald A. Norman. 2013. The Design of Everyday Things. Basic Books, Inc., New York, NY, USA.

Digital Library

[30]

Luke Olsen, Faramarz F. Samavati, Mario Costa Sousa, and Joaquim A. Jorge. 2009. Sketch-based modeling: A survey. Computers & Graphics 33, 1 (Feb. 2009), 85--103.

Digital Library

[31]

Chris De Paoli and Karan Singh. 2015. SecondSkin: sketch-based construction of layered 3D models. ACM Transactions on Graphics 34, 4 (July 2015), 126:1--126:10.

Digital Library

[32]

Robert Pastel. 2006. Measuring the Difficulty of Steering Through Corners. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1087--1096.

Digital Library

[33]

Huaishu Peng, Jimmy Briggs, Cheng-Yao Wang, Kevin Guo, Joseph Kider, Stefanie Mueller, Patrick Baudisch, and François Guimbretière. 2018. RoMA: Interactive Fabrication with Augmented Reality and a Robotic 3D Printer. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 579, 12 pages.

Digital Library

[34]

Alec Rivers, Andrew Adams, and Frédo Durand. 2012. Sculpting by numbers. ACM Transactions on Graphics 31, 6 (Nov. 2012), 157:1--157:7.

Digital Library

[35]

Johannes Schmid, Martin Sebastian Senn, Markus Gross, and Robert W. Sumner. 2011. OverCoat: an implicit canvas for 3D painting. In ACM SIGGRAPH 2011 papers on - SIGGRAPH '11. ACM Press.

Digital Library

[36]

Ryan Schmidt, Azam Khan, Karan Singh, and Gord Kurtenbach. 2009. Analytic drawing of 3D scaffolds. ACM Transactions on Graphics 28, 5 (Dec. 2009), 149:1--149:10.

Digital Library

[37]

Hyunyoung Song, François Guimbretière, Chang Hu, and Hod Lipson. 2006. ModelCraft: capturing freehand annotations and edits on physical 3D models. In Proceedings of the 19th annual ACM symposium on User interface software and technology - UIST '06. ACM Press.

Digital Library

[38]

Christian Weichel, Jason Alexander, Abhijit Karnik, and Hans Gellersen. 2015. SPATA: Spatio-Tangible Tools for Fabrication-Aware Design. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '14. ACM Press.

Digital Library

[39]

Christian Weichel, Manfred Lau, David Kim, Nicolas Villar, and Hans W. Gellersen. 2014. MixFab: a mixed-reality environment for personal fabrication. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14. ACM Press.

Digital Library

[40]

Gerold Wesche and Hans-Peter Seidel. 2001. FreeDrawer: a free-form sketching system on the responsive workbench. In Proceedings of the ACM symposium on Virtual reality software and technology - VRST '01. ACM Press.

Digital Library

[41]

Eva Wiese, Johann Habakuk Israel, Achim Meyer, and Sara Bongartz. 2010. Investigating the Learnability of Immersive Free-hand Sketching. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium (SBIM '10). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 135--142. http://dl.acm.org/citation.cfm?id=1923363.1923387

Digital Library

[42]

Min Xin, Ehud Sharlin, and Mario Costa Sousa. 2008. Napkin sketch: hand-held mixed reality 3D sketching. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology - VRST '08. ACM Press.

Digital Library

[43]

Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics 33, 4 (July 2014), 1--13.

Digital Library

[44]

Ya-Ting Yue, Xiaolong Zhang, Yongliang Yang, Gang Ren, Yi-King Choi, and Wenping Wang. 2017. WireDraw: 3D Wire Sculpturing Guided with Mixed Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems-CHI'17. ACM Press.

Digital Library

[45]

Kening Zhu, Alexandru Dancu, and Shengdong (Shen) Zhao. 2016. FusePrint: A DIY 2.5D Printing Technique Embracing Everyday Artifacts. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems - DIS '16. ACM Press.

Digital Library

Cited By

Wang SSarbach ARaubal M(2024)3D Flight Planning Using Extended RealityProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682088(1-10)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682088
Takeyama YNagayama KChoi MShizuki B(2024)PhoneCanvas: 3D Sketching System Using a Depth Camera-Equipped Smartphone as a CanvasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682078(1-12)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682078
Raina AMone VGorjian MQuek FSueda SKrishnamurthy V(2024)Blended Physical-Digital Kinesthetic Feedback for Mixed Reality-Based Conceptual Design-In-ContextProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670967(1-16)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670967
Show More Cited By

Index Terms

Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction paradigms
      1. Mixed / augmented reality

Recommendations

Experimental Evaluation of Sketching on Surfaces in VR
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Sketching in immersive 3D virtual reality (VR) environments has great potential for a variety of interactive 3D design applications. Precisely sketching the intended strokes in mid-air, however, can be a challenge. In this paper, we present a set of ...
Mobi3DSketch: 3D Sketching in Mobile AR
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Mid-air 3D sketching has been mainly explored in Virtual Reality (VR) and typically requires special hardware for motion capture and immersive, stereoscopic displays. The recently developed motion tracking algorithms allow real-time tracking of mobile ...
Physical Guides: An Analysis of 3D Sketching Performance on Physical Objects in Augmented Reality
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Augmented Reality (AR) lets users sketch 3D designs directly attached to existing physical objects. These objects provide natural haptic feedback whenever the pen touches them, and, unlike in VR, there is no need to digitize the physical object first. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SUI '18: Proceedings of the 2018 ACM Symposium on Spatial User Interaction

October 2018

203 pages

ISBN:9781450357081

DOI:10.1145/3267782

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

SUI '18

Sponsor:

SUI '18: Symposium on Spatial User Interaction

October 13 - 14, 2018

Berlin, Germany

Acceptance Rates

SUI '18 Paper Acceptance Rate 19 of 61 submissions, 31%;

Overall Acceptance Rate 86 of 279 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
706
Total Downloads

Downloads (Last 12 months)81
Downloads (Last 6 weeks)6

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang SSarbach ARaubal M(2024)3D Flight Planning Using Extended RealityProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682088(1-10)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682088
Takeyama YNagayama KChoi MShizuki B(2024)PhoneCanvas: 3D Sketching System Using a Depth Camera-Equipped Smartphone as a CanvasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682078(1-12)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682078
Raina AMone VGorjian MQuek FSueda SKrishnamurthy V(2024)Blended Physical-Digital Kinesthetic Feedback for Mixed Reality-Based Conceptual Design-In-ContextProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670967(1-16)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670967
Ying WHeo S(2024)Enhancing VR Sketching with a Dynamic Shape DisplayProceedings of the 30th ACM Symposium on Virtual Reality Software and Technology10.1145/3641825.3687714(1-11)Online publication date: 9-Oct-2024
https://dl.acm.org/doi/10.1145/3641825.3687714
Ying WRahman AHeo S(2024)Demonstrating VRScroll: A Shape-Changing Device for Precise Sketching in Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3648662(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3648662
Rodriguez RSullivan BBarrera Machuca MBatmaz ATornatzky COrtega F(2024)An Artists' Perspectives on Natural Interactions for Virtual Reality 3D SketchingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642758(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642758
Tan XHe ZLiu CFan MLuo TLiu ZTian MHan TTian F(2024)WieldingCanvas: Interactive Sketch Canvases for Freehand Drawing in VRProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642047(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642047
Machuca MIsrael JKeefe DStuerzlinger W(2024)Toward More Comprehensive Evaluations of 3D Immersive Sketching, Drawing, and PaintingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327629130:8(4648-4664)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3276291
Xiao TKim KKrukar JSubramaniyan RKiefer PSchwering ARaubal M(2024)VResin: Externalizing spatial memory into 3D sketch mapsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103322190(103322)Online publication date: Oct-2024
https://doi.org/10.1016/j.ijhcs.2024.103322
Bonneau GHahmann S(2024)3D sketching in immersive environmentsComputers and Graphics10.1016/j.cag.2024.103978123:COnline publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.103978
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents