More Web Proxy on the site http://driver.im/

research-article

TangibleData: Interactive Data Visualization with Mid-Air Haptics

Authors:

Ayush Bhardwaj,

Richard Huynh Noeske,

Jin Ryong KimAuthors Info & Claims

VRST '21: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology

Article No.: 29, Pages 1 - 11

https://doi.org/10.1145/3489849.3489890

Published: 08 December 2021 Publication History

Abstract

In this paper, we investigate the effects of mid-air haptics in interactive 3D data visualization. We build an interactive 3D data visualization tool that adapts hand gestures and mid-air haptics to provide tangible interaction in VR using ultrasound haptic feedback on 3D data visualization. We consider two types of 3D visualization datasets and provide different data encoding methods for haptic representations. Two user experiments are conducted to evaluate the effectiveness of our approach. The first experimental results show that adding a mid-air haptic modality can be beneficial regardless of noise conditions and useful for handling occlusion or discerning density and volume information. The second experiment results further show the strengths and weaknesses of direct touch and indirect touch modes. Our findings can shed light on designing and implementing a tangible interaction on 3D data visualization with mid-air haptic feedback.

References

[1]

Mehdi Ammi and Brian FG Katz. 2014. Intermodal audio-haptic metaphor: improvement of target search in abstract environments. International journal of human-computer interaction 30, 11(2014), 921–933.

[2]

Kei Aoyagi, Wen Wen, Qi An, Shunsuke Hamasaki, Hiroshi Yamakawa, Yusuke Tamura, Atsushi Yamashita, and Hajime Asama. 2021. Modified sensory feedback enhances the sense of agency during continuous body movements in virtual reality. Scientific reports 11, 1 (2021), 1–10.

[3]

Ricardo S Avila and Lisa M Sobierajski. 1996. A haptic interaction method for volume visualization. In Proceedings of Seventh Annual IEEE Visualization’96. IEEE, 197–204.

[4]

Benjamin Bach, Ronell Sicat, Johanna Beyer, Maxime Cordeil, and Hanspeter Pfister. 2017. The hologram in my hand: How effective is interactive exploration of 3d visualizations in immersive tangible augmented reality?IEEE transactions on visualization and computer graphics 24, 1(2017), 457–467.

[5]

Lonni Besançon, Paul Issartel, Mehdi Ammi, and Tobias Isenberg. 2016. Hybrid tactile/tangible interaction for 3D data exploration. IEEE transactions on visualization and computer graphics 23, 1(2016), 881–890.

[6]

Alberto Betella, Enrique Martínez Bueno, Wipawee Kongsantad, Riccardo Zucca, Xerxes D Arsiwalla, Pedro Omedas, and Paul FMJ Verschure. 2014. Understanding large network datasets through embodied interaction in virtual reality. In Proceedings of the 2014 Virtual Reality International Conference. 1–7.

Digital Library

[7]

Doug A Bowman and Chadwick A Wingrave. 2001. Design and evaluation of menu systems for immersive virtual environments. In Proceedings IEEE Virtual Reality 2001. IEEE, 149–156.

Digital Library

[8]

David J Brenner. 1993. Dose, volume, and tumor-control predictions in radiotherapy. International Journal of Radiation Oncology* Biology* Physics 26, 1(1993), 171–179.

[9]

Leandro Cancar, Alex Díaz, Antonio Barrientos, David Travieso, and David M Jacobs. 2013. Tactile-sight: A sensory substitution device based on distance-related vibrotactile flow. International Journal of Advanced Robotic Systems 10, 6 (2013), 272.

[10]

Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 505–514.

Digital Library

[11]

David J Cowperthwaite, M Sheelagh T Carpendale, and F David Fracchia. 1996. Visual access for 3D data. In Conference Companion on Human Factors in Computing Systems. 175–176.

Digital Library

[12]

Sanmathi Dangeti, Yingjie Victor Chen, and Chunhui Zheng. 2016. Comparing bare-hand-in-air Gesture and Object-in-hand Tangible User Interaction for Navigation of 3D Objects in Modeling. In Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. 417–421.

Digital Library

[13]

Niklas Elmqvist and Philippas Tsigas. 2008. A taxonomy of 3d occlusion management for visualization. IEEE transactions on visualization and computer graphics 14, 5(2008), 1095–1109.

Digital Library

[14]

Rickard Englund, Karljohan Lundin Palmerius, Ingrid Hotz, and Anders Ynnerman. 2018. Touching data: Enhancing visual exploration of flow data with haptics. Computing in Science & Engineering 20, 3 (2018), 89–100.

[15]

Barrett Ens, Benjamin Bach, Maxime Cordeil, Ulrich Engelke, Marcos Serrano, Wesley Willett, Arnaud Prouzeau, Christoph Anthes, Wolfgang Büschel, Cody Dunne, 2021. Grand challenges in immersive analytics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.

Digital Library

[16]

Adrien Fonnet and Yannick Prie. 2019. Survey of immersive analytics. IEEE transactions on visualization and computer graphics (2019).

[17]

LR Gavrilov and EM Tsirulnikov. 2012. Focused ultrasound as a tool to input sensory information to humans. Acoustical Physics 58, 1 (2012), 1–21.

[18]

Hyunjae Gil, Hyungki Son, Jin Ryong Kim, and Ian Oakley. 2018. Whiskers: Exploring the use of ultrasonic haptic cues on the face. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[19]

Keisuke Hasegawa and Hiroyuki Shinoda. 2013. A method for distribution control of aerial ultrasound radiation pressure for remote vibrotactile display. In The SICE Annual Conference 2013. IEEE, 223–228.

[20]

Ken Hinckley, Randy Pausch, John C Goble, and Neal F Kassell. 1994. Passive real-world interface props for neurosurgical visualization. In Proceedings of the SIGCHI conference on Human factors in computing systems. 452–458.

Digital Library

[21]

Christophe Hurter, Alexandru Telea, and Ozan Ersoy. 2011. Moleview: An attribute and structure-based semantic lens for large element-based plots. IEEE Transactions on Visualization and Computer Graphics 17, 12(2011), 2600–2609.

Digital Library

[22]

Inwook Hwang, Hyungki Son, and Jin Ryong Kim. 2017. AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic feedback. In 2017 IEEE World Haptics Conference (WHC). IEEE, 213–218.

[23]

Paul Issartel, Florimond Guéniat, and Mehdi Ammi. 2014. Slicing techniques for handheld augmented reality. In 2014 IEEE symposium on 3D user interfaces (3DUI). IEEE, 39–42.

[24]

Hiroo Iwata and Haruo Noma. 1993. Volume haptization. In Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium. IEEE, 16–23.

[25]

Bret Jackson, Tung Yuen Lau, David Schroeder, Kimani C Toussaint, and Daniel F Keefe. 2013. A lightweight tangible 3D interface for interactive visualization of thin fiber structures. IEEE transactions on visualization and computer graphics 19, 12(2013), 2802–2809.

Digital Library

[26]

Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2013. Evaluating the efficiency of physical visualizations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2593–2602.

Digital Library

[27]

Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæk. 2015. Opportunities and challenges for data physicalization. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 3227–3236.

Digital Library

[28]

Frederic Kaplan. 2009. Are gesture-based interfaces the future of human computer interaction?. In Proceedings of the 2009 international conference on Multimodal interfaces. 239–240.

Digital Library

[29]

Chaowanan Khundam. 2015. First person movement control with palm normal and hand gesture interaction in virtual reality. In 2015 12th International Joint Conference on Computer Science and Software Engineering (JCSSE). IEEE, 325–330.

[30]

Jin Ryong Kim, Stephanie Chan, Xiangchao Huang, Kenneth Ng, Limin Paul Fu, and Chen Zhao. 2019. Demonstration of refinity: an interactive holographic signage for new retail shopping experience. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–4.

Digital Library

[31]

Tijmen Klein, Florimond Guéniat, Luc Pastur, Frédéric Vernier, and Tobias Isenberg. 2012. A design study of direct-touch interaction for exploratory 3D scientific visualization. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 1225–1234.

[32]

Benjamin Long, Sue Ann Seah, Tom Carter, and Sriram Subramanian. 2014. Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1–10.

Digital Library

[33]

Christos Lougiakis, Akrivi Katifori, Maria Roussou, and Ioannis-Panagiotis Ioannidis. 2020. Effects of virtual hand representation on interaction and embodiment in HMD-based virtual environments using controllers. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 510–518.

[34]

Michael J McGuffin, Liviu Tancau, and Ravin Balakrishnan. 2003. Using deformations for browsing volumetric data. In IEEE Visualization, 2003. VIS 2003. IEEE, 401–408.

Digital Library

[35]

Bob Menelas, Mehdi Ammi, Luc Pastur, and Patrick Bourdot. 2009. Haptical exploration of an unsteady flow. In World Haptics 2009-Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. IEEE, 232–237.

Digital Library

[36]

Bob-Antoine J Menelas. 2014. Virtual Reality Technologies (Visual, Haptics, and Audio) in Large Datasets Analysis. In Innovative Approaches of Data Visualization and Visual Analytics. IGI Global, 111–132.

[37]

Harald Piringer, Robert Kosara, and Helwig Hauser. 2004. Interactive focus+ context visualization with linked 2d/3d scatterplots. In Proceedings. Second International Conference on Coordinated and Multiple Views in Exploratory Visualization, 2004. IEEE, 49–60.

Digital Library

[38]

Arnaud Prouzeau, Maxime Cordeil, Clement Robin, Barrett Ens, Bruce H Thomas, and Tim Dwyer. 2019. Scaptics and highlight-planes: Immersive interaction techniques for finding occluded features in 3d scatterplots. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[39]

Stefan Reifinger, Florian Laquai, and Gerhard Rigoll. 2008. Translation and rotation of virtual objects in Augmented Reality: A comparison of interaction devices. In 2008 IEEE International Conference on Systems, Man and Cybernetics. IEEE, 2448–2453.

[40]

Ted Romanus, Sam Frish, Mykola Maksymenko, William Frier, Loïc Corenthy, and Orestis Georgiou. 2019. Mid-air haptic bio-holograms in mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 348–352.

[41]

Isa Rutten, William Frier, and David Geerts. 2020. Discriminating Between Intensities and Velocities of Mid-Air Haptic Patterns. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 78–86.

[42]

Kittasil Silanon and Nikom Suvonvorn. 2011. Real time hand tracking as a user input device. In Knowledge, information, and creativity support systems. Springer, 178–189.

[43]

Jaemin Soh, Yeongjae Choi, Youngmin Park, and Hyun S Yang. 2013. User-friendly 3D object manipulation gesture using Kinect. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. 231–234.

Digital Library

[44]

Peng Song, Wooi Boon Goh, William Hutama, Chi-Wing Fu, and Xiaopei Liu. 2012. A handle bar metaphor for virtual object manipulation with mid-air interaction. In Proceedings of the SIGCHI conference on human factors in computing systems. 1297–1306.

Digital Library

[45]

Markus Straub, Andreas Riener, and Alois Ferscha. 2009. Distance encoding in vibro-tactile guidance cues. In 2009 6th Annual International Mobile and Ubiquitous Systems: Networking & Services, MobiQuitous. IEEE, 1–2.

[46]

Tomomi Takashina, Mitsuru Ito, and Yuji Kokumai. 2019. Midair Haptic Representation for Internal Structure in Volumetric Data Visualization. In SIGGRAPH Asia 2019 Posters. 1–2.

[47]

UltraLeap. [n.d.]. How does Ultrahaptics technology work. https://developer.ultrahaptics.com/knowledgebase/haptics-overview/

[48]

Paul Winkler, Philipp Stiens, Nadine Rauh, Thomas Franke, and Josef Krems. 2020. How latency, action modality and display modality influence the sense of agency: a virtual reality study. Virtual Reality 24, 3 (2020), 411–422.

Digital Library

[49]

Tae-Heon Yang, Jin Ryong Kim, Hanbit Jin, Hyunjae Gil, Jeong-Hoi Koo, and Hye Jin Kim. 2021. Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality. Advanced Functional Materials(2021), 2008831.

[50]

Nesra Yannier, Ali Israr, Jill Fain Lehman, and Roberta L Klatzky. 2015. FeelSleeve: Haptic feedback to enhance early reading. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 1015–1024.

Digital Library

[51]

Kazuma Yoshino and Hiroyuki Shinoda. 2014. Contactless touch interface supporting blind touch interaction by aerial tactile stimulation. In 2014 IEEE Haptics Symposium (HAPTICS). IEEE, 347–350.

[52]

Cha Zhang and Tsuhan Chen. 2001. Efficient feature extraction for 2D/3D objects in mesh representation. In Proceedings 2001 International Conference on Image Processing (Cat. No. 01CH37205), Vol. 3. IEEE, 935–938.

Cited By

Atzenbeck CEidloth L(2024)Harnessing Hypertext Paradigms to Augment VR SpacesProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688633(1-10)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1145/3679058.3688633
Eidloth LAtzenbeck CPfeiffer T(2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1145/3679058.3688632
Bhardwaj AKim J(2024)Tangible Data: Immersive Data Exploration with TouchACM SIGGRAPH 2024 Emerging Technologies10.1145/3641517.3664398(1-2)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641517.3664398
Show More Cited By

Recommendations

Tangible Data: Immersive Data Exploration with Touch
SIGGRAPH '24: ACM SIGGRAPH 2024 Emerging Technologies

We demonstrate an interactive 3D data visualization tool called TangibleData. TangibleData adapts hand gestures and mid-air haptics to provide 3D data exploration and interaction in VR using hand gestures and ultrasound mid-air haptic feedback. We ...
Designing Spatial Visualization and Interactions of Immersive Sankey Diagram in Virtual Reality
MM '24: Proceedings of the 32nd ACM International Conference on Multimedia

Virtual reality (VR) is a revolutionary method of presenting data visualizations, which brings potential possibilities for enhancing analytical activities. However, applying this method to visualize complex data flows remains largely underexplored, ...
Introduction to IATK: An Immersive Visual Analytics toolkit
ISS '19: Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces

Immersive Analytics is an emerging interdisciplinary research area that investigates the use of nontraditional display and input technology to immerse users in their data. A prominent aspect this research investigates is "immersive data visualization", ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

VRST '21: Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology

December 2021

563 pages

ISBN:9781450390927

DOI:10.1145/3489849

Editors:
Yuichi Itoh
Aoyama Gakuin University, Japan
,
Kazuki Takashima
Tohoku University, Japan
,
Parinya Punpongsanon
Osaka University, Japan
,
Misha Sra
University of California, Santa Barbara, USA
,
Kazuyuki Fujita
Tohoku University, Japan
,
Shigeo Yoshida
The University of Tokyo, Japan
,
Tham Piumsomboon
University of Canterbury, New Zealand

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 December 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

VRST '21

Sponsor:

VRST '21: 27th ACM Symposium on Virtual Reality Software and Technology

December 8 - 10, 2021

Osaka, Japan

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
765
Total Downloads

Downloads (Last 12 months)160
Downloads (Last 6 weeks)12

Reflects downloads up to 29 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Atzenbeck CEidloth L(2024)Harnessing Hypertext Paradigms to Augment VR SpacesProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688633(1-10)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1145/3679058.3688633
Eidloth LAtzenbeck CPfeiffer T(2024)Stepping into the Unknown: Immersive Spatial HypertextProceedings of the 7th Workshop on Human Factors in Hypertext10.1145/3679058.3688632(1-7)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1145/3679058.3688632
Bhardwaj AKim J(2024)Tangible Data: Immersive Data Exploration with TouchACM SIGGRAPH 2024 Emerging Technologies10.1145/3641517.3664398(1-2)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641517.3664398
Friedl‐Knirsch JPointecker FPfistermüller SStach CAnthes CRoth D(2024)A Systematic Literature Review of User Evaluation in Immersive AnalyticsComputer Graphics Forum10.1111/cgf.1511143:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15111
Bhardwaj AKim J(2024)Demonstration of TangibleData: Immersive Data Exploration with Touch2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct64951.2024.00192(646-647)Online publication date: 21-Oct-2024
https://doi.org/10.1109/ISMAR-Adjunct64951.2024.00192
Xu SGe XKaul CMurray-Smith R(2024)HpEIS: Learning Hand Pose Embeddings for Multimedia Interactive Systems2024 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME57554.2024.10688341(1-6)Online publication date: 15-Jul-2024
https://doi.org/10.1109/ICME57554.2024.10688341
Uhl JSchrom-Feiertag HRegal GHirsch LWeiss YTscheligi M(2023)When Realities Interweave: Exploring the Design Space of Immersive Tangible XRProceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction10.1145/3569009.3571843(1-4)Online publication date: 26-Feb-2023
https://dl.acm.org/doi/10.1145/3569009.3571843
Saffo DDi Bartolomeo SCrnovrsanin TSouth LRaynor JYildirim CDunne C(2023)Unraveling the Design Space of Immersive Analytics: A Systematic ReviewIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332736830:1(495-506)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3327368
Wojna KGeorgiou OBeattie DWright MLutteroth C(2023)Does It Par-?: Investigating the Relationship Between Mid-Air Haptics and Visual Representations of Surface TexturesIEEE Transactions on Haptics10.1109/TOH.2023.327295116:4(561-566)Online publication date: 4-May-2023
https://dl.acm.org/doi/10.1109/TOH.2023.3272951
Jang JFrier WPark J(2022)Multimodal Volume Data Exploration through Mid-Air Haptics2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR55827.2022.00039(243-251)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR55827.2022.00039
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten