More Web Proxy on the site http://driver.im/

research-article

Gaze-Adaptive Above and On-Surface Interaction

Authors:

Khalil Klouche,

Giulio JacucciAuthors Info & Claims

DIS '17: Proceedings of the 2017 Conference on Designing Interactive Systems

Pages 115 - 127

https://doi.org/10.1145/3064663.3064744

Published: 10 June 2017 Publication History

Abstract

We explore the combination of above-surface sensing with eye tracking to facilitate concurrent interaction with multiple regions on touch screens. Conventional touch input relies on positional accuracy, thereby requiring tight visual monitoring of one's own motor action. In contrast, above-surface sensing and eye tracking provides information about how user's hands and gaze are distributed across the interface. In these situations we facilitate interaction by 1) showing the visual feedback of the hand hover near user's gaze point and 2) decrease the requisite of positional accuracy by employing gestural information. We contribute input and visual feedback techniques that combine these modalities and demonstrate their use in example applications. A controlled study showed the effectiveness of our techniques for manipulation tasks against conventional touch, while the effectiveness in acquisition tasks depended on the amount of mid-air motion, leading to our conclusion that the techniques can benefit interacting with multiple interface regions.

References

[1]

1. Michelle Annett, Tovi Grossman, Daniel Wigdor, and George Fitzmaurice. 2011. Medusa: A Proximity-aware Multi-touch Tabletop. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 337--346.

Digital Library

[2]

2. Ilhan Aslan, Alina Krischkowsky, Alexander Meschtscherjakov, Martin Wuchse, and Manfred Tscheligi. 2015. A Leap for Touch: Proximity Sensitive Touch Targets in Cars. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '15). ACM, New York, NY, USA, 39--46.

Digital Library

[3]

3. Patrick Baudisch, Edward Cutrell, Dan Robbins, Mary Czerwinski, Peter Tandler, Benjamin Bederson, Alex Zierlinger, and others. 2003. Drag-and-pop and drag-and-pick: Techniques for accessing remote screen content on touch-and pen-operated systems. In Proceedings of INTERACT, Vol. 3. IOS Press, 57--64.

[4]

4. Hrvoje Benko, T. Scott Saponas, Dan Morris, and Desney Tan. 2009. Enhancing Input on and Above the Interactive Surface with Muscle Sensing. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '09). ACM, New York, NY, USA, 93--100.

Digital Library

[5]

5. Xiaojun Bi, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2011. Magic Desk: Bringing Multi-touch Surfaces into Desktop Work. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2511--2520.

Digital Library

[6]

6. Ishan Chatterjee, Robert Xiao, and Chris Harrison. 2015. Gaze+Gesture: Expressive, Precise and Targeted Free-Space Interactions. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI '15). ACM, New York, NY, USA, 131--138.

Digital Library

[7]

7. Xiang Chen, Julia Schwarz, Chris Harrison, Jennifer Mankoff, and Scott E. Hudson. 2014. Air+Touch: Interweaving Touch & In-air Gestures. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 519--525.

Digital Library

[8]

8. Victor Cheung, Jens Heydekorn, Stacey Scott, and Raimund Dachselt. 2012. Revisiting Hovering: Interaction Guides for Interactive Surfaces. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS '12). ACM, New York, NY, USA, 355--358.

Digital Library

[9]

9. Daewoong Choi, Hyeonjoong Cho, and Joono Cheong. 2015. Improving Virtual Keyboards When All Finger Positions Are Known. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, New York, NY, USA, 529--538.

Digital Library

[10]

10. Bruno De Araujo, Géry Casiez, and Joaquim Jorge. 2012. Mockup Builder: Direct 3D Modeling On and Above the Surface in a Continuous Interaction Space. In GI'12 38th Graphics Interface conference. Canadian Information Processing Society, Toronto, Canada, 173--180. https://hal.inria.fr/hal-00687891

Digital Library

[11]

11. George W. Fitzmaurice and William Buxton. 1997. An Empirical Evaluation of Graspable User Interfaces: Towards Specialized, Space-multiplexed Input. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 43--50.

Digital Library

[12]

12. Toshiyuki Hagiya and Tsuneo Kato. 2015. Probabilistic Touchscreen Text Entry Method Incorporating Gaze Point Information. Journal of Information Processing 23, 5 (2015), 708--715.

[13]

13. Otmar Hilliges, Shahram Izadi, Andrew D. Wilson, Steve Hodges, Armando Garcia-Mendoza, and Andreas Butz. 2009. Interactions in the Air: Adding Further Depth to Interactive Tabletops. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, New York, NY, USA, 139--148.

Digital Library

[14]

14. Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O'Hara, Gavin Smyth, and William Buxton. 2016. Pre-Touch Sensing for Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2869--2881.

Digital Library

[15]

15. M. Jeannerod. 1984. The Timing of Natural Prehension Movements. Journal of Motor Behavior 16, 3 (1984), 235--254.

[16]

16. Azam Khan, George Fitzmaurice, Don Almeida, Nicolas Burtnyk, and Gordon Kurtenbach. 2004. A Remote Control Interface for Large Displays. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST '04). ACM, New York, NY, USA, 127--136.

Digital Library

[17]

17. Nicolai Marquardt, Ricardo Jota, Saul Greenberg, and Joaquim A. Jorge. 2011. The Continuous Interaction Space: Interaction Techniques Unifying Touch and Gesture on and above a Digital Surface. Springer Berlin Heidelberg, Berlin, Heidelberg, 461--476.

[18]

18. Ken Pfeuffer, Jason Alexander, Ming Ki Chong, and Hans Gellersen. 2014. Gaze-touch: Combining Gaze with Multi-touch for Interaction on the Same Surface. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 509--518.

Digital Library

[19]

19. Ken Pfeuffer, Jason Alexander, Ming Ki Chong, Yanxia Zhang, and Hans Gellersen. 2015. Gaze-Shifting: Direct-Indirect Input with Pen and Touch Modulated by Gaze. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15). ACM, New York, NY, USA, 373--383.

Digital Library

[20]

20. Henning Pohl and Roderick Murray-Smith. 2013. Focused and Casual Interactions: Allowing Users to Vary Their Level of Engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2223--2232.

Digital Library

[21]

21. Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying Fixations and Saccades in Eye-tracking Protocols. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (ETRA '00). ACM, New York, NY, USA, 71--78.

Digital Library

[22]

22. Joey Scarr, Andy Cockburn, and Carl Gutwin. 2013. Supporting and Exploiting Spatial Memory in User Interfaces. Found. Trends Hum.-Comput. Interact. 6, 1 (Dec. 2013), 1--84.

Digital Library

[23]

23. Dominik Schmidt, Florian Block, and Hans Gellersen. 2009. A Comparison of Direct and Indirect Multi-touch Input for Large Surfaces. In Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I (INTERACT '09). Springer-Verlag, Berlin, Heidelberg, 582--594.

Digital Library

[24]

24. Julia Schwarz, Scott Hudson, Jennifer Mankoff, and Andrew D. Wilson. 2010. A Framework for Robust and Flexible Handling of Inputs with Uncertainty. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, New York, NY, USA, 47--56.

Digital Library

[25]

25. Baris Serim and Giulio Jacucci. 2016. Pointing While Looking Elsewhere: Designing for Varying Degrees of Visual Guidance During Manual Input. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5789--5800.

Digital Library

[26]

26. Will Spijkers and Simone Spellerberg. 1995. On-line Visual Control of Aiming Movements' Acta Psychologica 90, 1-3 (1995), 333--348. Discrete and Continuous Information Processing.

[27]

27. Sophie Stellmach and Raimund Dachselt. 2013. Still Looking: Investigating Seamless Gaze-supported Selection, Positioning, and Manipulation of Distant Targets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 285--294.

Digital Library

[28]

28. Philip Tuddenham, David Kirk, and Shahram Izadi. 2010. Graspables Revisited: Multi-touch vs. Tangible Input for Tabletop Displays in Acquisition and Manipulation Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2223--2232.

Digital Library

[29]

29. Jayson Turner, Jason Alexander, Andreas Bulling, Dominik Schmidt, and Hans Gellersen. 2013. Eye Pull, Eye Push: Moving Objects between Large Screens and Personal Devices with Gaze and Touch. Springer Berlin Heidelberg, Berlin, Heidelberg, 170--186.

[30]

30. Md. Sami Uddin, Carl Gutwin, and Benjamin Lafreniere. 2016. HandMark Menus: Rapid Command Selection and Large Command Sets on Multi-Touch Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5836--5848.

Digital Library

[31]

31. Simon Voelker, Andrii Matviienko, Johannes Schöning, and Jan Borchers. 2015. Combining Direct and Indirect Touch Input for Interactive Workspaces Using Gaze Input. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction (SUI '15). ACM, New York, NY, USA, 79--88.

Digital Library

[32]

32. Malte Weiss, Simon Voelker, Christine Sutter, and Jan Borchers. 2010. BendDesk: Dragging Across the Curve. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM, New York, NY, USA, 1--10.

Digital Library

[33]

33. Malte Weiss, Chat Wacharamanotham, Simon Voelker, and Jan Borchers. 2011. FingerFlux: Near-surface Haptic Feedback on Tabletops. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, New York, NY, USA, 615--620.

Digital Library

[34]

34. Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. 2009. SLAP Widgets: Bridging the Gap Between Virtual and Physical Controls on Tabletops. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 481--490.

Digital Library

[35]

35. Jinglong Wu, Jiajia Yang, and Taichi Honda. 2010. Fitts' law holds for pointing movements under conditions of restricted visual feedback. Human Movement Science 29, 6 (2010), 882--892.

[36]

36. Haijun Xia, Ricardo Jota, Benjamin McCanny, Zhe Yu, Clifton Forlines, Karan Singh, and Daniel Wigdor. 2014. Zero-latency Tapping: Using Hover Information to Predict Touch Locations and Eliminate Touchdown Latency. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 205--214.

Digital Library

[37]

37. Xing-Dong Yang, Tovi Grossman, Pourang Irani, and George Fitzmaurice. 2011. TouchCuts and TouchZoom: Enhanced Target Selection for Touch Displays Using Finger Proximity Sensing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 2585--2594.

Digital Library

[38]

38. Shumin Zhai, Carlos Morimoto, and Steven Ihde. 1999. Manual and Gaze Input Cascaded (MAGIC) Pointing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, New York, NY, USA, 246--253.

Digital Library

Cited By

Serim BPfeuffer KGellersen HJacucci G(2018)Visual Attention-Based AccessProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649432:3(1-23)Online publication date: 18-Sep-2018
https://dl.acm.org/doi/10.1145/3264943
Pauchet SLetondal CVinot JCausse MCousy MBecquet VCrouzet GKoskinen ILim YCerratto-Pargman TChow KOdom W(2018)GazeFormProceedings of the 2018 Designing Interactive Systems Conference10.1145/3196709.3196712(1193-1205)Online publication date: 8-Jun-2018
https://dl.acm.org/doi/10.1145/3196709.3196712
Westendorf LShaer OVarsanyi Pvan der Meulen HKun A(2017)Understanding Collaborative Decision Making Around a Large-Scale Interactive TabletopProceedings of the ACM on Human-Computer Interaction10.1145/31347451:CSCW(1-21)Online publication date: 6-Dec-2017
https://dl.acm.org/doi/10.1145/3134745
Show More Cited By

Index Terms

Gaze-Adaptive Above and On-Surface Interaction
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

GazeCast: Using Mobile Devices to Allow Gaze-based Interaction on Public Displays
ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

Gaze is promising for natural and spontaneous interaction with public displays, but current gaze-enabled displays require movement-hindering stationary eye trackers or cumbersome head-mounted eye trackers. We propose and evaluate GazeCast – a novel ...
Snap, Pursuit and Gain: Virtual Reality Viewport Control by Gaze
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

Head-mounted displays let users explore virtual environments through a viewport that is coupled with head movement. In this work, we investigate gaze as an alternative modality for viewport control, enabling exploration of virtual worlds with less head ...
Gaze and Touch Interaction on Tablets
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and Technology

We explore how gaze can support touch interaction on tablets. When holding the device, the free thumb is normally limited in reach, but can provide an opportunity for indirect touch input. Here we propose gaze and touch input, where touches redirect to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '17: Proceedings of the 2017 Conference on Designing Interactive Systems

June 2017

1444 pages

ISBN:9781450349222

DOI:10.1145/3064663

General Chair:
Oli Mival
Edinburgh Napier University, UK
,
Program Chairs:
Michael Smyth
Edinburgh Napier University, UK
,
Peter Dalsgaard
Aarhus University, Denmark

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Seventh Framework Programme

Conference

DIS '17

Sponsor:

SIGCHI

DIS '17: Designing Interactive Systems Conference 2017

June 10 - 14, 2017

Edinburgh, United Kingdom

Acceptance Rates

DIS '17 Paper Acceptance Rate 107 of 487 submissions, 22%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
262
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Serim BPfeuffer KGellersen HJacucci G(2018)Visual Attention-Based AccessProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649432:3(1-23)Online publication date: 18-Sep-2018
https://dl.acm.org/doi/10.1145/3264943
Pauchet SLetondal CVinot JCausse MCousy MBecquet VCrouzet GKoskinen ILim YCerratto-Pargman TChow KOdom W(2018)GazeFormProceedings of the 2018 Designing Interactive Systems Conference10.1145/3196709.3196712(1193-1205)Online publication date: 8-Jun-2018
https://dl.acm.org/doi/10.1145/3196709.3196712
Westendorf LShaer OVarsanyi Pvan der Meulen HKun A(2017)Understanding Collaborative Decision Making Around a Large-Scale Interactive TabletopProceedings of the ACM on Human-Computer Interaction10.1145/31347451:CSCW(1-21)Online publication date: 6-Dec-2017
https://dl.acm.org/doi/10.1145/3134745
Serim BChech LVasilieva MPapa LGamberini LJacucci GBiuk-Aghai RLi JTakahashi S(2017)Exploring gaze-adaptive features for interacting with multi-document visualizationsProceedings of the 10th International Symposium on Visual Information Communication and Interaction10.1145/3105971.3105977(85-92)Online publication date: 14-Aug-2017
https://dl.acm.org/doi/10.1145/3105971.3105977

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents