abstract

Towards multimodal interactions: robot jogging in mixed reality

Authors:

Trygve ThomessenAuthors Info & Claims

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

Article No.: 78, Pages 1 - 2

https://doi.org/10.1145/3139131.3141200

Published: 08 November 2017 Publication History

Get Access

Abstract

The recent progress made in the field of Augmented Reality/Mixed Reality (AR/MR) has opened new possibilities and approaches to research areas that can benefit from 3D visualization of digital content in the real world. In fact, human-robot interaction design and the design of user interfaces have very much to gain from MR technologies. Nonetheless, designing the user-robot interaction and processing multimodal feedbacks are very challenging tasks. In this paper we focus in particular on interactions in mixed reality.

The main contribution of this paper is the implementation of a control system for an industrial manipulator through the user's interactions with MR content displayed with the Microsoft HoloLens. The system is based on the communication between Unity3D (used to design the user experience) and ROS, therefore extendible to any ROS-compatible robotic hardware.

References

[1]

Jongeun Cha, Ian Oakley, Junhun Lee, and Jeha Ryu. 2005. An AR System for Haptic Communication. In Proceedings of the 2005 International Conference on Augmented Tele-existence (ICAT '05). ACM, New York, NY, USA, 241--242.

Digital Library

Google Scholar

[2]

Haiyang Jin, Liwei Zhang, S. Rockel, Jun Zhang, Ying Hu, and Jianwei Zhang. 2015. A novel optical tracking based tele-control system for tabletop object manipulation tasks. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 636--642.

Crossref

Google Scholar

[3]

Kai Kruckel, Florian Nolden, Alexander Ferrein, and Ingrid Scholl. 2015. Intuitive visual teleoperation for UGVs using free-look augmented reality displays. 4412--4417. http://opus.bibliothek.fh-aachen.de/frontdoor/index/index/docId/7412

Google Scholar

[4]

Dennis Krupke, Lasse Einig, Eike Langbehn, Jianwei Zhang, and Frank Steinicke. 2016. Immersive Remote Grasping: Realtime Gripper Control by a Heterogenous Robot Control System. In Proceedings of the 22Nd ACM Conference on Virtual Reality Software and Technology (VRST '16). ACM, New York, NY, USA, 337--338.

Digital Library

Google Scholar

[5]

L. Peppoloni, F. Brizzi, C. A. Avizzano, and E. Ruffaldi. 2015. Immersive ROS-integrated framework for robot teleoperation. In 2015 IEEE Symposium on 3D User Interfaces (3DUI). 177--178.

Crossref

Google Scholar

[6]

S. Starke, N. Hendrich, S. Magg, and J. Zhang. 2016. An efficient hybridization of Genetic Algorithms and Particle Swarm Optimization for inverse kinematics. In 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO). 1782--1789.

Crossref

Google Scholar

Cited By

View all

Dongye XWeng DJiang HTian ZBao YChen P(2024)Personalized decision-making for agents in face-to-face interaction in virtual realityMultimedia Systems10.1007/s00530-024-01591-731:1Online publication date: 24-Dec-2024
https://doi.org/10.1007/s00530-024-01591-7
Suzuki RKarim AXia THedayati HMarquardt N(2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517719
De Pace FSanna AManuri FOriti DPanicucci SBelcamino V(2022)Assessing the Effectiveness of Augmented Reality Handheld Interfaces for Robot Path ProgrammingIntelligent Technologies for Interactive Entertainment10.1007/978-3-030-99188-3_21(336-352)Online publication date: 25-Mar-2022
https://doi.org/10.1007/978-3-030-99188-3_21
Show More Cited By

Index Terms

Towards multimodal interactions: robot jogging in mixed reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces
      2. Mixed / augmented reality

Recommendations

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers. Augmented and mixed reality (AR/MR) have emerged as a new way to enhance human-robot interaction (HRI) and robotic interfaces (e.g., ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...
Virtual, Augmented, and Mixed Reality for HRI (VAM-HRI)
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction

The 5$^th $ International Workshop on Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) will bring together HRI, robotics, and mixed reality researchers to address challenges in mixed reality interactions between humans and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology

November 2017

437 pages

ISBN:9781450355483

DOI:10.1145/3139131

General Chairs:
Morten Fjeld
Chalmers University of Technology
,
Marco Fratarcangeli
Chalmers University of Technology
,
Daniel Sjölie
University of Gothenburg
,
Program Chairs:
Oliver Staadt
University of Rostock
,
Jonas Unger
Linköping University

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2017

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Norwegian Research Council

Conference

VRST '17

Sponsor:

VRST '17: 23rd ACM Symposium on Virtual Reality Software and Technology

November 8 - 10, 2017

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 66 of 254 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
302
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)3

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Dongye XWeng DJiang HTian ZBao YChen P(2024)Personalized decision-making for agents in face-to-face interaction in virtual realityMultimedia Systems10.1007/s00530-024-01591-731:1Online publication date: 24-Dec-2024
https://doi.org/10.1007/s00530-024-01591-7
Suzuki RKarim AXia THedayati HMarquardt N(2022)Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic InterfacesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517719(1-33)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517719
De Pace FSanna AManuri FOriti DPanicucci SBelcamino V(2022)Assessing the Effectiveness of Augmented Reality Handheld Interfaces for Robot Path ProgrammingIntelligent Technologies for Interactive Entertainment10.1007/978-3-030-99188-3_21(336-352)Online publication date: 25-Mar-2022
https://doi.org/10.1007/978-3-030-99188-3_21
Linjawi MMoore R(2018)Towards a Comprehensive Taxonomy for Characterizing RobotsTowards Autonomous Robotic Systems10.1007/978-3-319-96728-8_32(381-392)Online publication date: 21-Jul-2018
https://doi.org/10.1007/978-3-319-96728-8_32

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces

Multimodal augmented reality: the norm rather than the exception

Virtual, Augmented, and Mixed Reality for HRI (VAM-HRI)