More Web Proxy on the site http://driver.im/

research-article

Multimodal integration of natural gaze behavior for intention recognition during object manipulation

Authors:

Matthias Vogelgesang,

Edmund KlausAuthors Info & Claims

ICMI-MLMI '09: Proceedings of the 2009 international conference on Multimodal interfaces

Pages 199 - 206

https://doi.org/10.1145/1647314.1647350

Published: 02 November 2009 Publication History

Abstract

Naturally gaze is used for visual perception of our environment and gaze movements are mainly controlled subconsciously. Forcing the user to consciously diverge from that natural gaze behavior for interaction purposes causes high cognitive workload and destroys information contained in natural gaze movements. Instead of proposing a new gaze-based interaction technique, we analyze natural gaze behavior during an object manipulation task and show ways how it can be used for intention recognition, which provides a universal basis for integrating gaze into multimodal interfaces for different applications. We propose a model for multimodal integration of natural gaze behavior and evaluate it for two different use cases, namely for improvement of robustness of other potentially noisy input cues and for the design of proactive interaction techniques.

References

[1]

T. Bader, R. Rapple, and J. Beyerer. Fast invariant contour-based classification of hand symbols for hci. In CAIP 2009, volume 5702 of LNCS, pages 689--696. Springer, 2009.

Digital Library

[2]

S. K. Card, J. D. Mackinlay, and G. G. Robertson. The design space of input devices. In CHI '90: Proceedings of the conference on Human factors in computing systems, pages 117--124, New York, NY, USA, 1990. ACM Press.

Digital Library

[3]

A. T. Duchowski. Eye Tracking Methodology, Theory and Practice. Springer, 2003.

Digital Library

[4]

J. R. Flanagan and R. S. Johansson. Action plans used in action observation. Nature, 424:769--771, 2003.

[5]

B. Gesierich, A. Bruzzo, G. Ottoboni, and L. Finos. Human gaze behaviour during action execution and observation. Acta Psychologica, 128:324--330, 2008.

[6]

R. J. K. Jacob. What you look at is what you get: eye movement-based interaction techniques. In CHI '90: Proceedings of the conference on Human factors in computing systems, pages 11--18. ACM, 1990.

Digital Library

[7]

R. S. Johansson, G. Westling, A. Backstrom, and J. R. Flanagan. Eye-hand coordination in object manipulation. The Journal of Neuroscience, 21(17):6917--6932, 2001.

[8]

M. F. Land and D. N. Lee. Where we look when we steer. Nature, 369:742--744, 1994.

[9]

M. F. Land and P. McLeod. From eye movements to actions: how batsmen hit the ball. Nature Neuroscience, 3:1340--1345, 2000.

[10]

C. Lankford. E ective eye--gaze input into windows. In ETRA '00: Proceedings of the symposium on Eye tracking research&applications, pages 23--27, New York, NY, USA, 2000. ACM.

Digital Library

[11]

B. Law, M. S. Atkins, A. E. Kirkpatrick, and A. J. Lomax. Eye gaze patterns di erentiate novice and experts in a virtual laparoscopic surgery training environment. In ETRA '04: Proceedings of the symposium on Eye tracking research&applications, pages 41--48. ACM, 2004.

Digital Library

[12]

J. Pelz, M. M. Hayhoe, and R. Loeber. The coordination of eye, head, and hand movements in a natural task. Exp Brain Res, pages 266--277, 2001.

[13]

D. D. Salvucci and J. H. Goldberg. Identifying xations and saccades in eye--tracking protocols. In ETRA '00: Proceedings of the symposium on Eye tracking research&applications, pages 71--78. ACM, 2000.

Digital Library

[14]

SensoMotoric Instruments GmbH (SMI), Warthestrassee 21 D-14513 Teltow/Berlin. iView X Manual Version 1.03.09, September 2003.

[15]

B. A. Smith, J. Ho, W. Ark, and S. Zhai. Hand eye coordination patterns in target selection. In ETRA '00: Proceedings of the symposium on Eye tracking research&applications, pages 117--122. ACM, 2000.

Digital Library

Cited By

Severitt BCastner NWahl S(2024)Bi-Directional Gaze-Based Communication: A ReviewMultimodal Technologies and Interaction10.3390/mti81201088:12(108)Online publication date: 4-Dec-2024
https://doi.org/10.3390/mti8120108
Belardinelli A(2024)Gaze-Based Intention Estimation: Principles, Methodologies, and Applications in HRIACM Transactions on Human-Robot Interaction10.1145/365637613:3(1-30)Online publication date: 26-Sep-2024
https://dl.acm.org/doi/10.1145/3656376
Khan ANewn JBailey JVelloso E(2022)Integrating Gaze and Speech for Enabling Implicit InteractionsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502134(1-14)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502134
Show More Cited By

Index Terms

Multimodal integration of natural gaze behavior for intention recognition during object manipulation
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Simple gaze gestures and the closure of the eyes as an interaction technique
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

We created a set of gaze gestures that utilize the following three elements: simple one-segment gestures, off-screen space, and the closure of the eyes. These gestures are to be used as the moving tool in a gaze-only controlled drawing application. We ...
Snap, Pursuit and Gain: Virtual Reality Viewport Control by Gaze
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

Head-mounted displays let users explore virtual environments through a viewport that is coupled with head movement. In this work, we investigate gaze as an alternative modality for viewport control, enabling exploration of virtual worlds with less head ...
Comparing Dwell time, Pursuits and Gaze Gestures for Gaze Interaction on Handheld Mobile Devices
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Gaze is promising for hands-free interaction on mobile devices. However, it is not clear how gaze interaction methods compare to each other in mobile settings. This paper presents the first experiment in a mobile setting that compares three of the most ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI-MLMI '09: Proceedings of the 2009 international conference on Multimodal interfaces

November 2009

374 pages

ISBN:9781605587721

DOI:10.1145/1647314

General Chairs:
James L. Crowley
INRIA Grenoble Rhône-Alpes Research Centre, France
,
Yuri Ivanov
MERL, USA
,
Christopher Wren
Google, USA
,
Program Chairs:
Daniel Gatica-Perez
Idiap Research Institute, Switzerland
,
Michael Johnston
AT&T Research, USA
,
Rainer Stiefelhagen
University of Karlsruhe, Germany

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI-MLMI '09

Sponsor:

SIGCHI

ICMI-MLMI '09: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES/WORKSHOP ON MACHINE LEARNING FOR MULTIMODAL INTERFACES

November 2 - 4, 2009

Massachusetts, Cambridge, USA

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
521
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Severitt BCastner NWahl S(2024)Bi-Directional Gaze-Based Communication: A ReviewMultimodal Technologies and Interaction10.3390/mti81201088:12(108)Online publication date: 4-Dec-2024
https://doi.org/10.3390/mti8120108
Belardinelli A(2024)Gaze-Based Intention Estimation: Principles, Methodologies, and Applications in HRIACM Transactions on Human-Robot Interaction10.1145/365637613:3(1-30)Online publication date: 26-Sep-2024
https://dl.acm.org/doi/10.1145/3656376
Khan ANewn JBailey JVelloso E(2022)Integrating Gaze and Speech for Enabling Implicit InteractionsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502134(1-14)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502134
Petković TPetrović LMarković IPetrović I(2022)Human action prediction in collaborative environments based on shared-weight LSTMs with feature dimensionality reductionApplied Soft Computing10.1016/j.asoc.2022.109245126:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.asoc.2022.109245
Petković TPetrović LMarković IPetrović I(2022)Ensemble of LSTMs and Feature Selection for Human Action PredictionIntelligent Autonomous Systems 1610.1007/978-3-030-95892-3_33(429-441)Online publication date: 8-Apr-2022
https://doi.org/10.1007/978-3-030-95892-3_33
Gomez Cubero CRehm M(2021)Intention Recognition in Human Robot Interaction Based on Eye TrackingHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85613-7_29(428-437)Online publication date: 26-Aug-2021
https://doi.org/10.1007/978-3-030-85613-7_29
Siegfried RAminian BOdobez J(2020)ManiGaze: a Dataset for Evaluating Remote Gaze Estimator in Object Manipulation SituationsACM Symposium on Eye Tracking Research and Applications10.1145/3379156.3391369(1-5)Online publication date: 2-Jun-2020
https://dl.acm.org/doi/10.1145/3379156.3391369
Oh JPark JPark J(2019)Virtual Object Manipulation by Combining Touch and Head Interactions for Mobile Augmented RealityApplied Sciences10.3390/app91429339:14(2933)Online publication date: 22-Jul-2019
https://doi.org/10.3390/app9142933
Hu BLiu XWang WCai RLi FYuan S(2019)Prediction of interaction intention based on eye movement gaze feature2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC)10.1109/ITAIC.2019.8785737(378-383)Online publication date: May-2019
https://doi.org/10.1109/ITAIC.2019.8785737
Ravichandar HKumar ADani A(2018)Gaze and motion information fusion for human intention inferenceInternational Journal of Intelligent Robotics and Applications10.1007/s41315-018-0051-02:2(136-148)Online publication date: 19-Mar-2018
https://doi.org/10.1007/s41315-018-0051-0
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents