Preference in Voice Commands and Gesture Controls With Hands-Free Augmented Reality With Novel Users
Abstract
Hands-free augmented reality (HAR) has the potential of one day becoming our gateway to the Metaverse. However, the devices are still out of reach for most consumers mainly due to their price. The issues with novel interaction techniques can also present a challenge when interacting with holographic content in 3D space. In this article, we describe a mixed-method user study (N=34, inexperienced HAR users) with an augmented reality (AR) map application and HoloLens 2, where the target is to compare the usability, user experience (UX), and preferences with two of the most common HAR interaction techniques: gestures and voice commands. Our findings show that there may not be a significant difference in the overall UX between these controls when using an AR map. However, there are individual variability and differences in how novel users prefer to interact with holographic content. Gesture controls were preferred for their playfulness and voice commands for their efficiency. This suggests that developers should focus on different interaction techniques for different contexts from gaming to serious collaborative work.
References
[1]
T. Alsop, “AR headsets and glasses—Statistics & facts [Statistics],” Statista, 2023. [Online]. Available: https://www.statista.com/topics/10134/ar-glasses
[2]
R. Doerner et al., “Introduction to virtual and augmented reality,” in Virtual and Augmented Reality (VR/AR) Foundations and Methods of Extended Realities (XR). Cham, Switzerland: Springer, 2022, pp. 1–37.
[3]
R. Cheng, N. Wu, S. Chen, and B. Han, “Will metaverse be nextg internet? vision, hype, and reality,” IEEE Netw., vol. 36, no. 5, pp. 197–204, Sep./Oct. 2022.
[4]
S. Mystakidis, “Metaverse,” Encyclopedia, vol. 2, no. 1, pp. 486–497, 2022.
[5]
S. S. Muhammad Nizam et al., “A review of multimodal interaction technique in augmented reality environment,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 8, no. 4-2, 2018, Art. no.
[6]
H. J. Kang, J.-H. Shin, and K. Ponto, “A comparative analysis of 3D user interaction: How to move virtual objects in mixed reality,” in Proc. IEEE Conf. Virtual Reality 3D User Interfaces, 2020, pp. 275–284.
[7]
A. Khurshid et al., “Hand gesture recognition for user interaction in augmented reality (AR) experience,” in Virtual, Augmented and Mixed Reality: Design and Development: VAMR 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, 2022, pp. 306–316.
[8]
J. Toratti, “Integration of digital twin to augmented reality in industrial enrichment plants: An anticipated user experience study,” Master’s thesis, J. Toratti, University of Oulu, Oulu, Finland, 2022.
[9]
M. Prilla et al., “How to interact with augmented reality head-mounted devices in care work? A study comparing handheld touch (hands-on) and gesture (hands-free) interaction,” AIS Trans. Hum.- Comput. Interact., vol. 11, no. 3, pp. 157–178, 2019.
[10]
P. Bellani et al., “Enhancing user engagement in shared autonomous vehicles: An innovative gesture-based windshield interaction system,” Appl. Sci., vol. 13, no. 17, 2023, Art. no.
[11]
B. Pfleging et al., “Multimodal interaction in the car: Combining speech and gestures on the steering wheel,” in Proc. 4th Int. Conf. Automot. User Interfaces Interactive Veh. Appl., 2012, pp. 155–162.
[12]
K. Pfeuffer et al., “PalmGazer: Unimanual eye-hand menus in augmented reality,” 2023. [Online]. Available: https://arxiv.org/abs/2306.12402
[13]
M. Lee et al., “A usability study of multimodal input in an augmented reality environment,” Virtual Reality, vol. 17, pp. 293–305, 2013.
[14]
S. Sadri et al., “Manipulating 3D anatomic models in augmented reality: Comparing a hands-free approach and a manual approach,” in Proc. IEEE Int. Symp. Mixed Augmented Reality, 2019, pp. 93–102.
[15]
K. A. Satriadi, B. Ens, M. Cordeil, B. Jenny, T. Czauderna, and W. Willett, “Augmented reality map navigation with freehand gestures,” in Proc. IEEE Conf. Virtual Reality 3D User Interfaces, 2019 pp. 593–603.
[16]
Microsoft, “HoloLens research mode,” 2023. [Online]. Available: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/research-mode
[17]
H. J. Guo and B. Prabhakaran, “HoloLens 2 technical evaluation as mixed reality guide,” 2022. [Online]. Available: https://arxiv.org/abs/2207.09554
[18]
K. S. Hale et al., “Augmented reality sandtable (ARES) impacts on learning,” Hum. Factors Ergonom. Soc. Annu. Meeting, vol. 63, no. 1, pp. 2149–2153, 2019.
[19]
A. Pukari et al., “LeARn flags: Assessment of the learning curve of inexperienced users with gesture interactions with hands-free augmented reality,” in Proc. 26th Int. Academic Mindtrek Conf., 2023, pp. 340–343.
[20]
Brooke, “SUS- A quick and dirty usability scale,” Usability Eval. Ind., vol. 189, no. 194, pp. 4–7, 1996.
[21]
B. Laugwitz et al., “Construction and evaluation of a user experience questionnaire,” in HCI and Usability for Education and Work: USAB 2008, Berlin Heidelberg: Springer, 2008, pp. 63–76.
[22]
V. Braun and V. Clarke, “Using thematic analysis in psychology,” Qualitative Res. Psychol., vol. 3, no. 2, pp. 77–101, 2006.
[23]
K. Malterud et al., “Sample size in qualitative interview studies: Guided by information power,” Qualitative Health Res., vol. 26, no. 13, pp. 1753–1760, 2016.
[24]
K. Lebeck, K. Ruth, T. Kohno, and F. Roesner, “Securing augmented reality output,” in Proc. IEEE Symp. Secur. Privacy, 2017, pp. 320–337.
[25]
U. Riedlinger et al., “Tango vs. HoloLens: A comparison of collaborative indoor AR visualisations using hand-held and hands-free devices,” Multimodal Technol. Interact., vol. 3, no. 2, p. 23, 2019.
[26]
Finnish National Board on Research Integrity,“ The ethical principles of research with human participants and ethical review in the human sciences in Finland,” 2020. [Online]. Available: https://tenk.fi/sites/default/files/2021-01/Ethical_review_in_human_sciences_2020.pdf
[27]
University of Oulu,“ Ethics committee of human sciences,” 2024. [Online]. Available: https://www.oulu.fi/en/university/faculties-and-units/eudaimonia-institute/ethics-committee-human-sciences
Recommendations
Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
LeARn Flags: Assessment of the learning curve of inexperienced users with gesture interactions with hands-free augmented reality
Mindtrek '23: Proceedings of the 26th International Academic Mindtrek ConferenceHands-free augmented reality (HAR) is intrinsically different from mobile augmented reality (MAR), as the interactions with the digital or holographic content happen with HAR in 3D space. For novel users, this method of interaction can be difficult to ...
Free-hands interaction in augmented reality
SUI '13: Proceedings of the 1st symposium on Spatial user interactionThe ability to use free-hand gestures is extremely important for mobile augmented reality applications. This paper proposes a computer vision-driven model for natural free-hands interaction in augmented reality. The novelty of the research is the use of ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 04 March 2024
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in