More Web Proxy on the site http://driver.im/

research-article

Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input Technique

Authors:

Tanya R. Jonker,

Ben LafreniereAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 6, Issue 1

Article No.: 15, Pages 1 - 24

https://doi.org/10.1145/3517262

Published: 29 March 2022 Publication History

Abstract

Emerging input techniques that rely on sensing and recognition can misinterpret a user's intention, resulting in errors and, potentially, a negative user experience. To enhance the development of such input techniques, it is valuable to understand implications of these errors, but they can very costly to simulate. Through two controlled experiments, this work explores various low-cost methods for evaluating error acceptability of freehand mid-air gestural input in virtual reality. Using a gesture-driven game and a drawing application, the first experiment elicited error characteristics through text descriptions, video demonstrations, and a touchscreen-based interactive simulation. The results revealed that video effectively conveyed the dynamics of errors, whereas the interactive modalities effectively reproduced the user experience of effort and frustration. The second experiment contrasts the interactive touchscreen simulation with the target modality - a full VR simulation - and highlights the relative costs and benefits for assessment in an alternative, but still interactive, modality. These findings introduce a spectrum of low-cost methods for evaluating recognition-based errors in VR and a series of characteristics that can be understood in each.

Supplementary Material

henderson (henderson.zip)

Supplemental movie, appendix, image and software files for, Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input Technique

Download
96.39 KB

References

[1]

Fraser Anderson, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2013. YouMove: Enhancing Movement Training with an Augmented Reality Mirror. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST '13). Association for Computing Machinery, New York, NY, USA, 311--320. https://doi.org/10.1145/2501988.2502045

Digital Library

[2]

Ravin Balakrishnan and I. Scott MacKenzie. 1997. Performance Differences in the Fingers, Wrist, and Forearm in Computer Input Control. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI '97). Association for Computing Machinery, New York, NY, USA, 303--310. https://doi.org/10.1145/258549.258764

Digital Library

[3]

Olivier Bau and Wendy E. Mackay. 2008. OctoPocus: a dynamic guide for learning gesture-based command sets. In Proceedings of the 21st annual ACM symposium on User interface software and technology (Monterey, CA, USA, 2008-10-19) (UIST '08). Association for Computing Machinery, 37--46. https://doi.org/10.1145/1449715.1449724

Digital Library

[4]

Michel Beaudouin-Lafon and Wendy E. Mackay. 2012. Prototyping Tools and Techniques. In The Human Computer Interaction Handbook (3 ed.), Julie A. Jacko (Ed.). CRC Press, Boca Raton, FL, Chapter 47, 1081--1104. https://doi.org/10.1201/b11963-55

[5]

Marion Buchenau and Jane Fulton Suri. [n.d.]. Experience prototyping. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (New York City, New York, USA, 2000-08-01) (DIS '00). Association for Computing Machinery, 424--433. https://doi.org/10.1145/347642.347802

Digital Library

[6]

Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz Studies: Why and How. In Proceedings of the 1st International Conference on Intelligent User Interfaces (Orlando, Florida, USA) (IUI '93). Association for Computing Machinery, New York, NY, USA, 193--200. https://doi.org/10.1145/169891.169968

Digital Library

[7]

Fred Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13 (09 1989), 319-. https://doi.org/10.2307/249008

Digital Library

[8]

William Delamare, Thomas Janssoone, Céline Coutrix, and Laurence Nigay. 2016. Designing 3D Gesture Guidance: Visual Feedback and Feedforward Design Options. In Proceedings of the International Working Conference on Advanced Visual Interfaces (Bari, Italy) (AVI '16). Association for Computing Machinery, New York, NY, USA, 152--159. https://doi.org/10.1145/2909132.2909260

Digital Library

[9]

P. Dourish. 2004. Where the Action Is: The Foundations of Embodied Interaction. MIT Press. https://books.google.ca/books?id=-TRWc0PA9e4C

[10]

Graham Dove, Kim Halskov, Jodi Forlizzi, and John Zimmerman. [n.d.]. UX Design Innovation: Challenges for Working with Machine Learning As a Design Material. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (New York, NY, USA, 2017) (CHI '17). ACM, 278--288. https://doi.org/10.1145/3025453.3025739

Digital Library

[11]

Pelle Ehn and Morten Kyng. 1992. Cardboard Computers: Mocking-It-up or Hands-on the Future. In Design at Work: Cooperative Design of Computer Systems. L. Erlbaum Associates Inc., USA, 169--196.

[12]

World Leaders in Research-Based User Experience. 1994. Guerrilla HCI: Article by Jakob Nielsen. https://www.nngroup.com/articles/guerrilla-hci/ Library Catalog: www.nngroup.com.

[13]

World Leaders in Research-Based User Experience. 2009. Discount Usability: 20 Years. https://www.nngroup.com/articles/discount-usability-20-years/ Library Catalog: www.nngroup.com.

[14]

Dustin Freeman, Hrvoje Benko, Meredith Ringel Morris, and Daniel Wigdor. [n.d.]. ShadowGuides: visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Banff, Alberta, Canada, 2009-11-23) (ITS '09). Association for Computing Machinery, 165--172. https://doi.org/10.1145/1731903.1731935

Digital Library

[15]

Dustin Freeman, Hrvoje Benko, Meredith Ringel Morris, and Daniel Wigdor. 2009. ShadowGuides: Visualizations for in-Situ Learning of Multi-Touch and Whole-Hand Gestures. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (Banff, Alberta, Canada) (ITS '09). Association for Computing Machinery, New York, NY, USA, 165--172. https://doi.org/10.1145/1731903.1731935

Digital Library

[16]

Saul Greenberg and Chester Fitchett. [n.d.]. Phidgets: easy development of physical interfaces through physical widgets. In Proceedings of the 14th annual ACM symposium on User interface software and technology (Orlando, Florida, 2001-11-11) (UIST '01). Association for Computing Machinery, 209--218. https://doi.org/10.1145/502348.502388

Digital Library

[17]

Carl Gutwin, Andy Cockburn, and Benjamin Lafreniere. 2015. Testing the Rehearsal Hypothesis with Two FastTap Interfaces. 223--231.

[18]

Sandra G. Hart. 2006. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (2006), 904--908. https://doi.org/10.1177/154193120605000909 arXiv:https://doi.org/10.1177/154193120605000909

[19]

Björn Hartmann, Leith Abdulla, Manas Mittal, and Scott R. Klemmer. 2007. Authoring Sensor-Based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '07). Association for Computing Machinery, New York, NY, USA, 145--154. https://doi.org/10.1145/1240624.1240646

Digital Library

[20]

Jay Henderson, Sachi Mizobuchi, Wei Li, and Edward Lank. 2019. Exploring Cross-Modal Training via Touch to Learn a Mid-Air Marking Menu Gesture Set. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (Taipei, Taiwan) (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, Article 8, 9 pages. https://doi.org/10.1145/3338286.3340119

Digital Library

[21]

Ankit Kamal, Yang Li, and Edward Lank. [n.d.]. Teaching motion gestures via recognizer feedback. In Proceedings of the 19th international conference on Intelligent User Interfaces (Haifa, Israel, 2014-02-24) (IUI '14). Association for Computing Machinery, 73--82. https://doi.org/10.1145/2557500.2557521

Digital Library

[22]

Ankit Kamal, Yang Li, and Edward Lank. 2014. Teaching Motion Gestures via Recognizer Feedback. In Proceedings of the 19th International Conference on Intelligent User Interfaces (Haifa, Israel) (IUI '14). Association for Computing Machinery, New York, NY, USA, 73--82. https://doi.org/10.1145/2557500.2557521

Digital Library

[23]

Keiko Katsuragawa, Ankit Kamal, and Edward Lank. 2017. Effect of motion-gesture recognizer error pattern on user workload and behavior. In International Conference on Intelligent User Interfaces, Proceedings IUI. https://doi.org/10.1145/3025171.3025234

Digital Library

[24]

Keiko Katsuragawa, Ankit Kamal, Qi Feng Liu, Matei Negulescu, and Edward Lank. 2019. Bi-Level Thresholding: Analyzing the Effect of Repeated Errors in Gesture Input. ACM Trans. Interact. Intell. Syst. 9, 2-3, Article 15 (April 2019), 30 pages. https://doi.org/10.1145/3181672

Digital Library

[25]

Matthew Kay, Shwetak N. Patel, and Julie A. Kientz. 2015. How good is 85%? A survey tool to connect classifier evaluation to acceptability of accuracy. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/2702123.2702603

Digital Library

[26]

Matthew Kay and Jacob O. Wobbrock. 2020. Aligned Rank Transform for Nonparametric Factorial ANOVAs. https://doi.org/10.5281/zenodo.594511

[27]

Rafal Kocielnik, Saleema Amershi, and Paul N. Bennett. 2019. Will you accept an imperfect AI? Exploring Designs for Adjusting End-user Expectations of AI Systems. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300641

Digital Library

[28]

Matthias Kranz, Paul Holleis, and Albrecht Schmidt. 2010. Embedded Interaction: Interacting with the Internet of Things. IEEE Internet Computing 14, 2 (2010), 46--53. https://doi.org/10.1109/MIC.2009.141

Digital Library

[29]

Ben Lafreniere, Tanya R. Jonker, Stephanie Santosa, Mark Parent, Michael Glueck, Tovi Grossman, Hrvoje Benko, and Daniel Wigdor. 2021. False Positives vs. False Negatives: The Effects of Recovery Time and Cognitive Costs on Input Error Preference. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST '21). Association for Computing Machinery, New York, NY, USA, 54--68. https://doi.org/10.1145/3472749.3474735

Digital Library

[30]

David Ledo, Fraser Anderson, Ryan Schmidt, Lora Oehlberg, Saul Greenberg, and Tovi Grossman. [n.d.]. Pineal: Bringing Passive Objects to Life with Embedded Mobile Devices. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA, 2017-05-02) (CHI '17). Association for Computing Machinery, 2583--2593. https://doi.org/10.1145/3025453.3025652

Digital Library

[31]

David Ledo, Jo Vermeulen, Sheelagh Carpendale, Saul Greenberg, Lora Oehlberg, and Sebastian Boring. [n.d.]. Astral: Prototyping Mobile and Smart Object Interactive Behaviours Using Familiar Applications. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, CA, USA, 2019-06-18) (DIS '19). Association for Computing Machinery, 711--724. https://doi.org/10.1145/3322276.3322329

Digital Library

[32]

Yang Li. [n.d.]. Protractor: a fast and accurate gesture recognizer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA, 2010-04-10) (CHI '10). Association for Computing Machinery, 2169--2172. https://doi.org/10.1145/1753326.1753654

Digital Library

[33]

Microsoft. 2020. Gestures - Mixed Reality Toolkit Documentation. https://microsoft.github.io/MixedRealityToolkit-Unity/Documentation/Input/Gestures.html

[34]

William Odom, John Zimmerman, Scott Davidoff, Jodi Forlizzi, Anind K. Dey, and Min Kyung Lee. [n.d.]. A fieldwork of the future with user enactments. In Proceedings of the Designing Interactive Systems Conference (Newcastle Upon Tyne, United Kingdom, 2012-06-11) (DIS 12). Association for Computing Machinery, 338--347. https://doi.org/10.1145/2317956.2318008

Digital Library

[35]

Quentin Roy, Futian Zhang, and Daniel Vogel. 2019. Automation accuracy is good, but high controllability may be better. In Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3290605.3300750

Digital Library

[36]

Dean Rubine. 1991. Specifying Gestures by Example. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '91). Association for Computing Machinery, New York, NY, USA, 329--337. https://doi.org/10.1145/122718.122753

Digital Library

[37]

Valkyrie Savage, Colin Chang, and Björn Hartmann. 2013. Sauron: Embedded Single-Camera Sensing of Printed Physical User Interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST '13). Association for Computing Machinery, New York, NY, USA, 447--456. https://doi.org/10.1145/2501988.2501992

Digital Library

[38]

Rajinder Sodhi, Hrvoje Benko, and Andrew Wilson. 2012. LightGuide: Projected Visualizations for Hand Movement Guidance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI '12). Association for Computing Machinery, New York, NY, USA, 179--188. https://doi.org/10.1145/2207676.2207702

Digital Library

[39]

Seiichi Uchida and Hiroaki Sakoe. 2005. A survey of elastic matching techniques for handwritten character recognition. IEICE transactions on information and systems 88, 8 (2005), 1781--1790.

[40]

Viswanath Venkatesh and Fred Davis. 2000. A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies. Management Science 46 (02 2000), 186--204. https://doi.org/10.1287/mnsc.46.2.186.11926

[41]

Daniel S. Weld and Gagan Bansal. 2019. The Challenge of Crafting Intelligible Intelligence. Commun. ACM 62, 6 (May 2019), 70--79. https://doi.org/10.1145/3282486

Digital Library

[42]

Daniel Wigdor and Dennis Wixon. 2011. Brave NUI World: Designing Natural User Interfaces for Touch and Gesture (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[43]

Andrew D. Wilson. 2006. Robust Computer Vision-Based Detection of Pinching for One and Two-Handed Gesture Input. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (Montreux, Switzerland) (UIST 06). Association for Computing Machinery, New York, NY, USA, 255--258. https://doi.org/10.1145/1166253.1166292

Digital Library

[44]

Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI '11). Association for Computing Machinery, New York, NY, USA, 143--146. https://doi.org/10.1145/1978942.1978963

Digital Library

[45]

J. Zimmerman. 2005. Video sketches: exploring pervasive computing interaction designs. IEEE Pervasive Computing 4, 4 (2005), 91--94.

Digital Library

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Wang XLafreniere BZhao J(2024)Exploring Visualizations for Precisely Guiding Bare Hand Gestures in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642935(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642935

Index Terms

Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input Technique
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies

Recommendations

Exploring the use of hand-to-face input for interacting with head-worn displays
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We propose the use of Hand-to-Face input, a method to interact with head-worn displays (HWDs) that involves contact with the face. We explore Hand-to-Face interaction to find suitable techniques for common mobile tasks. We evaluate this form of ...
False Positives vs. False Negatives: The Effects of Recovery Time and Cognitive Costs on Input Error Preference
UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology

Existing approaches to trading off false positive versus false negative errors in input recognition are based on imprecise ideas of how these errors affect user experience that are unlikely to hold for all situations. To inform dynamic approaches to ...
Investigating current techniques for opposite-hand smartwatch interaction
MobileHCI '17: Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services

The small display size of smartwatches creates a challenge for touch input, which is still the interaction technique of choice. Researchers and producers have started to investigate alternative interaction techniques. Apple and Samsung, for example, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 6, Issue 1

March 2022

1009 pages

EISSN:2474-9567

DOI:10.1145/3529514

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 March 2022

Published in IMWUT Volume 6, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
269
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)4

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kianpisheh MMariakakis ATruong K(2024)exHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435008:1(1-30)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643500
Wang XLafreniere BZhao J(2024)Exploring Visualizations for Precisely Guiding Bare Hand Gestures in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642935(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642935

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents