More Web Proxy on the site http://driver.im/

demonstration

Remapped Interfaces: Building Contextually Adaptive User Interfaces with Haptic Retargeting

Authors:

Brandon J Matthews,

Carolin Reichherzer,

Ross T SmithAuthors Info & Claims

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 469, Pages 1 - 4

https://doi.org/10.1145/3544549.3583912

Published: 19 April 2023 Publication History

Abstract

User interfaces in virtual reality are dynamic and contextual, making it challenging to provide accurate haptic feedback. This paper presents a remapped interface system that leverages haptic retargeting to reuse a limited set of controls for a wide array of interfaces. The system enables the creation of virtual user interfaces that can adapt to the context in which they are being used and provide accurate haptic feedback. Using the system, we developed four exemplar applications: an immersive painting app, an extensible music production tool, a simulated aircraft cockpit and a user-configurable interface design tool.

Supplementary Material

VTT File (3544549.3583912-preview.vtt)

Download
.71 KB

VTT File (3544549.3583912-walkthrough.vtt)

Download
3.95 KB

MP4 File (3544549.3583912-walkthrough.mp4)

Walkthrough Video

Download
108.66 MB

MP4 File (3544549.3583912-preview.mp4)

Video Preview

Download
19.11 MB

References

[1]

Mahdi Azmandian, Mark Hancock, Hrvoje Benko, Eyal Ofek, and Andrew D. Wilson. 2016. Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI ’16. ACM Press, 1968–1979. https://doi.org/10.1145/2858036.2858226

Digital Library

[2]

Yuki Ban, Takuji Narumi, Tomohiro Tanikawa, and Michitaka Hirose. 2014. Displaying shapes with various types of surfaces using visuo-haptic interaction. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology(VRST ’14). Association for Computing Machinery, New York, NY, USA, 191–196. https://doi.org/10.1145/2671015.2671028

Digital Library

[3]

Li-Wei Chan, Hui-Shan Kao, Mike Y. Chen, Ming-Sui Lee, Jane Hsu, and Yi-Ping Hung. 2010. Touching the void: direct-touch interaction for intangible displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’10). Association for Computing Machinery, New York, NY, USA, 2625–2634. https://doi.org/10.1145/1753326.1753725

Digital Library

[4]

Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17. ACM Press, 3718–3728. https://doi.org/10.1145/3025453.3025753

Digital Library

[5]

Changseok Cho, Huichul Yang, G.J. Kim, and S.H. Han. 2002. Body-based interfaces. In Proceedings. Fourth IEEE International Conference on Multimodal Interfaces. 466–472. https://doi.org/10.1109/ICMI.2002.1167040

Digital Library

[6]

Cathy Mengying Fang and Chris Harrison. 2021. Retargeted Self-Haptics for Increased Immersion in VR without Instrumentation. In The 34th Annual ACM Symposium on User Interface Software and Technology(UIST ’21). Association for Computing Machinery, New York, NY, USA, 1109–1121. https://doi.org/10.1145/3472749.3474810

Digital Library

[7]

Martin Feick, Niko Kleer, André Zenner, Anthony Tang, and Antonio Krüger. 2021. Visuo-haptic Illusions for Linear Translation and Stretching using Physical Proxies in Virtual Reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems(CHI ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3411764.3445456

Digital Library

[8]

J. J. Gibson. 1933. Adaptation, after-effects, and contrast in the perception of curved lines. Journal of Experimental Psycholology 16, 1 (1933), 1–31. https://doi.org/10.1037/h0074626

[9]

L Kohli and M Whitton. 2005. The haptic hand: providing user interface feedback with the non-dominant hand in virtual environments. Proceedings of Graphics Interface 2005(2005), 1–8. https://doi.org/10.1145/1089508.1089510

[10]

R.W. Lindeman, J.L. Sibert, and J.N. Templeman. 2001. The effect of 3D widget representation and simulated surface constraints on interaction in virtual environments. In Proceedings IEEE Virtual Reality 2001. 141–148. https://doi.org/10.1109/VR.2001.913780

[11]

Brandon Matthews, Bruce H Thomas, Stewart Von Itzstein, and Ross Smith. 2021. Adaptive Reset Techniques for Haptic Retargeted Interaction. IEEE Transactions on Visualization and Computer Graphics (2021), 1–1. https://doi.org/10.1109/TVCG.2021.3120410 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[12]

Brandon J. Matthews and Ross T. Smith. 2019. Head Gaze Target Selection for Redirected Interaction. In SIGGRAPH Asia 2019 XR(SA ’19). Association for Computing Machinery, Brisbane, QLD, Australia, 13–14. https://doi.org/10.1145/3355355.3361883 event-place: Brisbane, QLD, Australia.

Digital Library

[13]

Brandon J. Matthews, Bruce H. Thomas, G. Stewart Von Itzstein, and Ross T. Smith. 2022. Shape Aware Haptic Retargeting for Accurate Hand Interactions. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 625–634. https://doi.org/10.1109/VR51125.2022.00083 ISSN: 2642-5254.

[14]

Brandon J. Matthews, Bruce H. Thomas, Stewart Von Itzstein, and Ross T. Smith. 2019. Remapped Physical-Virtual Interfaces with Bimanual Haptic Retargeting. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 19–27. https://doi.org/10.1109/VR.2019.8797974 ISSN: 2642-5254.

[15]

Markus Tatzgern and Christoph Birgmann. 2021. Exploring Input Approximations for Control Panels in Virtual Reality. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). 1–9. https://doi.org/10.1109/VR50410.2021.00092 ISSN: 2642-5254.

[16]

André Zenner and Antonio Krüger. 2019. Estimating Detection Thresholds for Desktop-Scale Hand Redirection in Virtual Reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 47–55. https://doi.org/10.1109/VR.2019.8798143 ISSN: 2642-5254.

[17]

Yiwei Zhao and Sean Follmer. 2018. A Functional Optimization Based Approach for Continuous 3D Retargeted Touch of Arbitrary, Complex Boundaries in Haptic Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, 1–12. https://doi.org/10.1145/3173574.3174118

Digital Library

Cited By

van Deurzen BRovelo Ruiz GBot DVanacken DLuyten K(2024)Substitute Buttons: Exploring Tactile Perception of Physical Buttons for Use as Haptic ProxiesMultimodal Technologies and Interaction10.3390/mti80300158:3(15)Online publication date: 20-Feb-2024
https://doi.org/10.3390/mti8030015
Plümer JTatzgern M(2023)Towards a Framework for Validating XR Prototyping for Performance Evaluations of Simulated User Experiences2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00096(810-819)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00096

Index Terms

Remapped Interfaces: Building Contextually Adaptive User Interfaces with Haptic Retargeting
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Perception
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Haptic devices
    2. Interaction paradigms
      1. Virtual reality

Recommendations

Towards Applied Remapped Physical-Virtual Interfaces: Synchronization Methods for Resolving Control State Conflicts
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

User interfaces in virtual reality enable diverse interactions within the virtual world, though they typically lack the haptic cues provided by physical interface controls. Haptic retargeting enables flexible mapping between dynamic virtual interfaces ...
Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Manipulating a virtual object with appropriate passive haptic cues provides a satisfying sense of presence in virtual reality. However, scaling such experiences to support multiple virtual objects is a challenge as each one needs to be accompanied with ...
3D User Interfaces: New Directions and Perspectives

Three-dimensional user interfaces (3D UIs) let users interact with virtual objects, environments, or information using direct 3D input in the physical and/or virtual space. In this article, the founders and organizers of the IEEE Symposium on 3D User ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

3914 pages

ISBN:9781450394222

DOI:10.1145/3544549

Editors:
Albrecht Schmidt
LMU Munich, Germany
,
Kaisa Väänänen
Tampere University, Finland
,
Tesh Goyal
Google Research, USA
,
Per Ola Kristensson
University of Cambridge, UK
,
Anicia Peters
University of Namibia, Namibia

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

Qualifiers

Demonstration
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
272
Total Downloads

Downloads (Last 12 months)141
Downloads (Last 6 weeks)30

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

van Deurzen BRovelo Ruiz GBot DVanacken DLuyten K(2024)Substitute Buttons: Exploring Tactile Perception of Physical Buttons for Use as Haptic ProxiesMultimodal Technologies and Interaction10.3390/mti80300158:3(15)Online publication date: 20-Feb-2024
https://doi.org/10.3390/mti8030015
Plümer JTatzgern M(2023)Towards a Framework for Validating XR Prototyping for Performance Evaluations of Simulated User Experiences2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00096(810-819)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00096

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents