More Web Proxy on the site http://driver.im/

research-article

Bringing the Scene Back to the Tele-operator: Auditory Scene Manipulation for Tele-presence Systems

Authors:

Carlos T. Ishi,

Hiroshi IshiguroAuthors Info & Claims

HRI '15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction

Pages 279 - 286

https://doi.org/10.1145/2696454.2696494

Published: 02 March 2015 Publication History

Abstract

In a tele-operated robot system, the reproduction of auditory scenes, conveying 3D spatial information of sound sources in the remote robot environment, is important for the transmission of remote presence to the tele-operator. We proposed a tele-presence system which is able to reproduce and manipulate the auditory scenes of a remote robot environment, based on the spatial information of human voices around the robot, matched with the operator's head orientation. In the robot side, voice sources are localized and separated by using multiple microphone arrays and human tracking technologies, while in the operator side, the operator's head movement is tracked and used to relocate the spatial positions of the separated sources. Interaction experiments with humans in the robot environment indicated that the proposed system had significantly higher accuracy rates for perceived direction of sounds, and higher subjective scores for sense of presence and listenability, compared to a baseline system using stereo binaural sounds obtained by two microphones located at the humanoid robot's ears. We also proposed three different user interfaces for augmented auditory scene control. Evaluation results indicated higher subjective scores for sense of presence and usability in two of the interfaces (control of voice amplitudes based on virtual robot positioning, and amplification of voices in the frontal direction).

References

[1]

Nishio, S., Ishiguro, H., Hagita, N. Can a Teleoperated Android Represent Personal Presence' - A Case Study with Children. Psychologia, 50(4): 330--342. 2007.

[2]

Ishi, C.T., Liu, C., Ishiguro, H., Hagita, N. 2010. Head motion during dialogue speech and nod timing control in humanoid robots. In Proceedings of 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2010). OSAKA, JAPAN. 293--300.

Digital Library

[3]

Liu, C., Ishi, C. T., Ishiguro, H., Hagita, N. Generation of nodding, head tilting and eye gazing for human-robot dialogue interaction. In Proceeding of ACM/IEEE International Conference on Human Robot Interaction (HRI 2012). Boston, USA. 285--292, March, 2012.

Digital Library

[4]

Sumioka, H., Nishio, S., Minato, T., Yamazaki, R., Ishiguro, H. Minimal Human Design Approach for Sonzai-kan Media: Investigation of a Feeling of Human Presence. Cognitive Computation, 2014.

[5]

Popescu, V. G., Burdea, G. C., Bouzit, M., Hentz, V. R. A virtual-reality-based telerehabilitation system with force feedback. IEEE transactions on Information Technology in Biomedicine. 4(1): 45--51. 2000.

Digital Library

[6]

Piron, L., Turolla, A., Agostini, M., Zucconi, C., Cortese, F., Zampolini, M., Zannini, M., Dam, M., Ventura, L., Battauz, M., Tonin, P. Exercises for paretic upper limb after stroke: a combined virtual-reality and telemedicine approach. J. of Rehabilitation Medicine. 41(12): 1016--1020(5). 2009.

[7]

Billinghurst, M., Cheok, A., Prince, S., Kato, H. Real world teleconferencing. IEEE Computer Graphics and Applications. 22(6): 11--13. 2002.

Digital Library

[8]

Ogi, T., Yamada, T., Tamagawa, K., Kano, M. Immersive telecommunication using stereo video avatar. Proceedings of Ieee Virtual Reality. Yokohama, Japan. 45--51. 2001

Digital Library

[9]

Bullinger, H., Riedel, O., Breining, R. Immersive Projection Technology- Benefits for the Industry, International Immersive Projection Technology Workshop, 13--25, 1997.

[10]

Pulkki, V. Spatial sound reproduction with directional audio coding. J. Audio Eng. Soc. 55(6): 503--516. 2007

[11]

Laitinen, M., Kuech, F., Disch, S., Pulkki, V. Reproducing applause-type signals with directional audio coding. J. Audio Eng. Soc. 59(1/2): 29--43. 2011.

[12]

Nishiura, T., Yamada, T., Nakamura, S., Shikano, K. Localization of multiple sound sources based on a CSP analysis with a microphone array. Proceeding of ICASSP 2000. Istanbul. II1053--1056 vol.2. 2000.

Digital Library

[13]

Khalil, F., Jullien, J. P., Gilloire, A. Microphone array for sound pickup in teleconference systems. J. Audio Eng. Soc. 42(9): 691--700. 1994.

[14]

Hamalainen, M., Myllyla, V. Acoustic Echo Cancellation for dynamically steered microphone array system. 2007. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, NY, USA. 58--61. 2007.

[15]

Rumsey, F. Spatial Audio. Focal Press, 2001.

[16]

Meyer, E., Neumann, E. Physical and Applied Acoustics: An Introduction. Academic Press, New York, 1972. ISBN 0124931502.

[17]

Cheng, C. I., Wakefield, G. H. Introduction to head-related transfer functions (hrtfs): Representations of hrtfs in time, frequency, and space. J. Acoust. Soc. Am, 49(4):231--249, April 2001.

[18]

Iwaya, Y., Suzuki, Y., Kimura, D. Effects of head movement on front-back error in sound localization. Acoustical Science and Technology. 24(5): 322--324. 2003.

[19]

Perret, S., Noble, W. The effect of head rotations on vertical plane sound localization. J. Acoust. Soc. Am. 102(4): 2325--2332. 1997.

[20]

Schmidt, R. Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas and Propagation, 34, 276--280, 1986.

[21]

Ishi, C. T., Chatot, O., Ishiguro, H., Hagita, N. Evaluation of a MUSIC-based real-time sound localization of multiple sound sources in real noisy environments. In Proceeding of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 09). St. Louis, MO, USA. 2027--2032. 2009.

Digital Library

[22]

Glas, D.F. et al, 2007. Laser tracking of human body motion using adaptive shape modeling. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007), 602--608. 2007.

[23]

Ishi, C., Even, J., Hagita, N. (2013). Using multiple microphone arrays and reflections for 3D localization of sound sources. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013), 3937--3942, Nov., 2013.

[24]

Dudgeon, D. E. Fundamentals of digital array processing. Proceedings of the IEEE. 65(6): 898--904. 1977.

[25]

Gardner, W. G., Martin, K. D. HRTF measurements of a KEMAR. J. Acoust. Soc. Am. 97(6):3907--3908, Jun. 1995.

[26]

Langton, S. R., Watt, R. J., Bruce, I. I. Do the eyes have it' Cues to the direction of social attention. Trends Cog. Sci. 4, 50--59, 2000.

[27]

Yokoyama, T., Noguchi, Y. Kita, S. Attentional shifts by gaze direction in voluntary orienting: evidence from a microsaccade study. Exp. Brain Res. 223, 291--300, 2012.

Cited By

Edirisinghe SGalagedarage ODias IRanaweera C(2023)Recent Development of Emerging Indoor Wireless Networks towards 6GNetwork10.3390/network30200143:2(269-297)Online publication date: 12-May-2023
https://doi.org/10.3390/network3020014
Schneiders ECheon EKjeldskov JRehm MSkov M(2022)Non-Dyadic Interaction: A Literature Review of 15 Years of Human-Robot Interaction Conference PublicationsACM Transactions on Human-Robot Interaction10.1145/348824211:2(1-32)Online publication date: 8-Feb-2022
https://dl.acm.org/doi/10.1145/3488242
Arakawa TTakase HIshida HSugimoto THorii SKamachi THoshuyama O(2022)Local-Sound Visualizations for Presence Control of Telepresence Robots2022 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE53296.2022.9730341(1-5)Online publication date: 7-Jan-2022
https://doi.org/10.1109/ICCE53296.2022.9730341
Show More Cited By

Index Terms

Bringing the Scene Back to the Tele-operator: Auditory Scene Manipulation for Tele-presence Systems
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
2. Computing methodologies
  1. Artificial intelligence
    1. Control methods
      1. Robotic planning
    2. Planning and scheduling
      1. Robotic planning

Recommendations

A Study on Realistic Audio Sound Generation according to User's Movement in Virtual Reality System
ICIIT '19: Proceedings of the 2019 4th International Conference on Intelligent Information Technology

Generally, a virtual reality (VR) system cannot provide the realistic sound formed by the multi-channel audio signals to a user with the stereo headphone environment. In addition, the VR system has a gap between the visual scene and the sound because it ...
A Tele-operated Humanoid Operator

This is the first successful trial of remotely controlling a humanoid robot to drive an industrial vehicle in lieu of a human operator. These results were achieved through the development of three technologies: 1) remote-control technology to instruct ...
A novel biologically inspired neural network solution for robotic 3D sound source sensing

This paper presents a novel real-time robotic binaural sound localization method based on hierarchical fuzzy artificial neural networks and a generic set of head related transfer functions. The robot is a humanoid equipped with the KEMAR artificial head ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction

March 2015

368 pages

ISBN:9781450328838

DOI:10.1145/2696454

General Chairs:
Julie A. Adams
Vanderbilt University, USA
,
William Smart
Oregon State University, USA
,
Program Chairs:
Bilge Mutlu
University of Wisconsin-Madison, USA
,
Leila Takayama
Google[x], USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
RA: IEEE Robotics and Automation Society
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 March 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

MIC/SCOPE
JST/CREST

Conference

HRI '15

Sponsor:

HRI '15: ACM/IEEE International Conference on Human-Robot Interaction

March 2 - 5, 2015

Oregon, Portland, USA

Acceptance Rates

HRI '15 Paper Acceptance Rate 43 of 169 submissions, 25%;

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
217
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)2

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Edirisinghe SGalagedarage ODias IRanaweera C(2023)Recent Development of Emerging Indoor Wireless Networks towards 6GNetwork10.3390/network30200143:2(269-297)Online publication date: 12-May-2023
https://doi.org/10.3390/network3020014
Schneiders ECheon EKjeldskov JRehm MSkov M(2022)Non-Dyadic Interaction: A Literature Review of 15 Years of Human-Robot Interaction Conference PublicationsACM Transactions on Human-Robot Interaction10.1145/348824211:2(1-32)Online publication date: 8-Feb-2022
https://dl.acm.org/doi/10.1145/3488242
Arakawa TTakase HIshida HSugimoto THorii SKamachi THoshuyama O(2022)Local-Sound Visualizations for Presence Control of Telepresence Robots2022 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE53296.2022.9730341(1-5)Online publication date: 7-Jan-2022
https://doi.org/10.1109/ICCE53296.2022.9730341
Ishi CLiu CEven JHagita N(2018)A sound-selective hearing support system using environment sensor networkAcoustical Science and Technology10.1250/ast.39.28739:4(287-294)Online publication date: 1-Jul-2018
https://doi.org/10.1250/ast.39.287
Ishi CLiu CEven JHagita N(2016)Hearing support system using environment sensor network2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS.2016.7759211(1275-1280)Online publication date: Oct-2016
https://doi.org/10.1109/IROS.2016.7759211

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents