More Web Proxy on the site http://driver.im/

research-article

Quick Bootstrapping of a Personalized Gaze Model from Real-Use Interactions

Authors:

Michael Xuelin Huang,

Hong Va LeongAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 9, Issue 4

Article No.: 43, Pages 1 - 25

https://doi.org/10.1145/3156682

Published: 30 January 2018 Publication History

Abstract

Understanding human visual attention is essential for understanding human cognition, which in turn benefits human--computer interaction. Recent work has demonstrated a Personalized, Auto-Calibrating Eye-tracking (PACE) system, which makes it possible to achieve accurate gaze estimation using only an off-the-shelf webcam by identifying and collecting data implicitly from user interaction events. However, this method is constrained by the need for large amounts of well-annotated data. We thus present fast-PACE, an adaptation to PACE that exploits knowledge from existing data from different users to accelerate the learning speed of the personalized model. The result is an adaptive, data-driven approach that continuously “learns” its user and recalibrates, adapts, and improves with additional usage by a user. Experimental evaluations of fast-PACE demonstrate its competitive accuracy in iris localization, validity of alignment identification between gaze and interactions, and effectiveness of gaze transfer. In general, fast-PACE achieves an initial visual error of 3.98 degrees and then steadily improves to 2.52 degrees given incremental interaction-informed data. Our performance is comparable to state-of-the-art, but without the need for explicit training or calibration. Our technique addresses the data quality and quantity problems. It therefore has the potential to enable comprehensive gaze-aware applications in the wild.

References

[1]

Seung Jin Baek, Kang A. Choi, Chunfei Ma, Young Hyun Kim, and Sung Jea Ko. 2013. Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems. IEEE Transactions on Consumer Electronics 59 (2013), 415--421.

[2]

Jixu Chen, Xiaoming Liu, and Amy Aragones. 2013. Learning person-specific models for facial expression and action unit recognition. Pattern Recognition Letters 34, 15 (November 2013), 1964--1970.

Digital Library

[3]

Yiu Ming Cheung and Qinmu Peng. 2015. Eye gaze tracking with a web camera in a desktop environment. IEEE Transactions on Human-Machine Syst. (2015).

[4]

Timothy F. Cootes, Gareth J. Edwards, and Christopher J. Taylor. 2001. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 6 (June 2001), 681--685.

Digital Library

[5]

Timothy F. Cootes and Christopher J. Taylor. 1992. Active shape models: “Smart snakes.” In BMVC92. 266--275.

[6]

Ribel Fares, Shaomin Fang, and Oleg Komogortsev. 2013. Can we beat the mouse with MAGIC? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). New York: ACM Press, 1387.

Digital Library

[7]

Martin A. Fischler and Robert C. Bolles. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 24 (1981), 381--395.

Digital Library

[8]

Anjith George and Aurobinda Routray. 2016. A fast and accurate algorithm for eye localization for gaze tracking in low resolution images. IET Computer Vision (2016).

[9]

Dan Witzner Hansen and Qiang Ji. 2010. In the eye of the beholder: A survey of models for eyes and gaze. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 3 (2010), 478--500.

Digital Library

[10]

Anthony J. Hornof and Tim Halverson. 2002. Cleaning up systematic error in eye-tracking data by using required fixation locations. Behavior Research Methods and Instrumental Computing 34 (2002), 592--604.

[11]

Jeff Huang, Ryen White, and Georg Buscher. 2012. User see, user point: gaze and cursor alignment in web search. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI’12). New York: ACM Press, 1341.

Digital Library

[12]

Jeff Huang, Ryen W. White, and Susan Dumais. 2011. No clicks, no problem: Using cursor movements to understand and improve search. In Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI’11). New York: ACM Press, 1225.

Digital Library

[13]

Michael Xuelin Huang, Tiffany C. K. Kwok, Grace Ngai, Stephen C. F. Chan, and Hong Va Leong. 2016. Building a personalized, auto-calibrating eye tracker from user interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 5169--5179.

Digital Library

[14]

Michael Xuelin Huang, Tiffany C. K. Kwok, Grace Ngai, Hong Va Leong, and Stephen C. F. Chan. 2014. Building a self-learning eye gaze model from user interaction data. In Proceeding of the ACM International Conference on Multimedia (MM’14), 1017--1020.

Digital Library

[15]

Michael Xuelin Huang, Jiajia Li, Grace Ngai, and Hong Va Leong. 2017. ScreenGlint: Practical, in-situ gaze estimation on smartphones. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI’17). New York: ACM Press, 2546--2557.

Digital Library

[16]

Michael Xuelin Huang, Jiajia Li, Grace Ngai, and Hong Va Leong. 2016. StressClick: Sensing stress from gaze-click patterns. In Proceedings of ACM International Conference on Multimedia. 1395--1404.

Digital Library

[17]

Qiong Huang, Ashok Veeraraghavan, and Ashutosh Sabharwal. 2017. TabletGaze: Unconstrained appearance-based gaze estimation in mobile tablets. Machine Vision and Applications 28, 5--6 (2017), 445--461.

Digital Library

[18]

Robert J. K. Jacob. 1990. What you look at is what you get: Eye movement-based interaction techniques. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People (CHI’90). New York: ACM Press, 11--18.

Digital Library

[19]

Oliver Jesorsky, Klaus J. Kirchberg, and Robert W. Frischholz. 2001. Robust face detection using the hausdorff distance. International Conference On Audio- and Video-based Biometric Person Authentication. Springer, 90--95.

Digital Library

[20]

Kyle Krafka, Aditya Khosla, Petr Kellnhofer, and Harini Kannan. 2015. Eye tracking for everyone.

[21]

Daniel J. Liebling and Susan T. Dumais. 2014. Gaze and mouse coordination in everyday work. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp’14 Adjunct), 1141--1150.

Digital Library

[22]

Feng Lu, Takahiro Okabe, Yusuke Sugano, and Yoichi Sato. 2014. Learning gaze biases with head motion for head pose-free gaze estimation. Image and Visual Computing 32, 3 (March 2014), 169--179.

Digital Library

[23]

Feng Lu, Yusuke Sugano, Takahiro Okabe, and Yoichi Sato. 2015. Gaze estimation from eye appearance: A head pose-free method via eye image synthesis. IEEE Transactions on Image Processing 24 (2015), 3680--3693.

Digital Library

[24]

Nenad Markuš, Miroslav Frljak, Igor S. Pandžić, Jörgen Ahlberg, and Robert Forchheimer. 2014. Eye pupil localization with an ensemble of randomized trees. Pattern Recognition 47 (2014), 578--587.

Digital Library

[25]

Sinno Jialin Pan and Qiang Yang. 2010. A survey on transfer learning. IEEE Transactions on Knowlege Data Engineering 22 (2010), 1345--1359.

Digital Library

[26]

Alexandra Papoutsaki, Patsorn Sangkloy, James Laskey, Nediyana Daskalova, Jeff Huang, and James Hays. 2016. WebGazer: Scalable webcam eye tracking using user interactions. In Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI). 3839--3845.

Digital Library

[27]

Michael Jones and Paul Viola. 2004. Robust real-time object detection. International Journal of Computer Vision 57, 2 (2004), 137--154.

Digital Library

[28]

Victoria Ponz, Arantxa Villanueva, and Rafael Cabeza. 2012. Dataset for the evaluation of eye detector for gaze estimation. Proceedings of the 2012 ACM Conference On Ubiquitous Computing (UbiComp’12), 681.

Digital Library

[29]

Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. “GrabCut” : Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics 23 (2004), 309.

Digital Library

[30]

Enver Sangineto, Gloria Zen, Elisa Ricci, and Nicu Sebe. 2014. We are not all equal: Personalizing models for facial expression analysis with transductive parameter transfer. In Proceedings of the ACM International Conference on Multimedia (MM’14). 357--366.

Digital Library

[31]

S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy. 2000. Improvements to the SMO algorithm for SVM regression. IEEE Transactions on Neural Networks 11 (2000), 1188--1193.

Digital Library

[32]

Yusuke Sugano, Yasuyuki Matsushita, and Yoichi Sato. 2013. Appearance-based gaze estimation using visual saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2 (February 2013), 329--41.

Digital Library

[33]

Yusuke Sugano, Yasuyuki Matsushita, Yoichi Sato, and Hideki Koike. 2008. An incremental learning method for unconstrained gaze estimation. In Proceedings of the 10th European Conference on Computer Vision (ECCV 2008). 656--667.

Digital Library

[34]

Yusuke Sugano, Yasuyuki Matsushita, Yoichi Sato, and Hideki Koike. 2015. Appearance-based gaze estimation with online calibration from mouse operations. IEEE Transactions on Human-Machine Systems 45, 6 (2015), 750--760.

[35]

Yusuke Sugano, Xucong Zhang, and Andreas Bulling. 2016. AggreGaze: Collective estimation of audience attention on public displays. Proceedings of the 29th Annual Symposium on User Interface Software (Technol UIST’16), 821--831.

Digital Library

[36]

Roberto Valenti and Theo Gevers. 2012. Accurate eye center location through invariant isocentric patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (2012), 1785--1798.

Digital Library

[37]

Roberto Valenti, Nicu Sebe, and Theo Gevers. 2012. Combining head pose and eye location information for gaze estimation. IEEE Transactions on Image Process. 21 (2012), 802--815.

Digital Library

[38]

Mélodie Vidal, Andreas Bulling, and Hans Gellersen. 2013. Pursuits: Spontaneous interaction with displays based on smooth pursuit eye movement and moving targets. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp’13). 439.

Digital Library

[39]

Erroll Wood, Tadas Baltrušaitis, Louis-Philippe Morency, Peter Robinson, and Andreas Bulling. 2016. Learning an appearance-based gaze estimator from one million synthesised images. In Proceedings of the 9th Biennial ACM Symposium on Eye Tracking Research 8 Applications (ETRA’16). 131--138.

Digital Library

[40]

Erroll Wood, Tadas Baltrusaitis, Xucong Zhang, Yusuke Sugano, Peter Robinson, and Andreas Bulling. 2015. Rendering of eyes for eye-shape registration and gaze estimation. In IEEE International Conference on Computer Vision (ICCV). 3756--3764.

Digital Library

[41]

Erroll Wood and Andreas Bulling. 2014. EyeTab: Model-based gaze estimation on unmodified tablet computers. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’14). New York: ACM Press, 207--210.

Digital Library

[42]

Xuehan Xiong and Fernando De la Torre. 2013. Supervised descent method and its applications to face alignment. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 532--539.

Digital Library

[43]

Chang Xu, Dacheng Tao, and Chao Xu. 2015. Multi-view intact space learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 12 (December 2015), 2531--2544.

Digital Library

[44]

P. Xu, K. A. Ehinger, and Y. Zhang. 2015. TurkerGaze: Crowdsourcing saliency with webcam based eye tracking. arXiv Prepr. arXiv (2015), 5.

[45]

Yi Yao and Gianfranco Doretto. 2010. Boosting for transfer learning with multiple sources. In Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE, 1855--1862.

[46]

Jun Yu, Yong Rui, and Dacheng Tao. 2014. Click prediction for web image reranking using multimodal sparse coding. IEEE Transactions on Image Process. 23, 5 (2014), 2019--2032.

[47]

Jun Yu, Dacheng Tao, Meng Wang, and Yong Rui. 2015. Learning to rank using user clicks and visual features for image retrieval. IEEE Transactions on Cybernetics 45, 4 (2015), 767--779.

[48]

Jun Yu, Xiaokang Yang, Fei Gao, and Dacheng Tao. 2016. Deep multimodal distance metric learning using click constraints for image ranking. IEEE Transactions on Cybernetics (2016), 1--11.

[49]

Gloria Zen, Lorenzo Porzi, Enver Sangineto, Elisa Ricci, and Nicu Sebe. 2016. Learning personalized models for facial expression analysis and gesture recognition. IEEE Transactions on Multimedia 18 (2016), 775--788.

[50]

Chi Zhang, Xiang Sun, Jiani Hu, and Weihong Deng. 2014. Precise eye localization by fast local linear SVM. In Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME). 1--6.

[51]

Lefei Zhang, Qian Zhang, Liangpei Zhang, Dacheng Tao, Xin Huang, and Bo Du. 2015. Ensemble manifold regularized sparse low-rank approximation for multiview feature embedding. Pattern Recognition 48, 10 (October 2015), 3102--3112.

Digital Library

[52]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2015. Appearance-based gaze estimation in the wild. In Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015). 9.

[53]

Xucong Zhang, Yusuke Sugano, Mario Fritz, and Andreas Bulling. 2017. It's written all over your face: Full-face appearance-based gaze estimation. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 2299--2308.

[54]

Mingcai Zhou, Xiying Wang, Haitao Wang, Jingu Heo, and DongKyung Nam. 2015. Precise eye localization with improved SDM. In Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP). 4466--4470.

[55]

Jie Zhu and Jie Yang. 2002. Subpixel eye gaze tracking. In Proceedings of the 5th IEEE International Conference on Automatic Face Gesture Recognition. 131--136.

Digital Library

Cited By

Bisogni CNappi MTortora GDel Bimbo A(2024)Gaze analysisImage and Vision Computing10.1016/j.imavis.2024.104961144:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.imavis.2024.104961
Fu EHuang MLeong HNgai GBoll SMu Lee KLuo JZhu WByun HWen Chen CLienhart RMei T(2018)Cross-Species LearningProceedings of the 26th ACM international conference on Multimedia10.1145/3240508.3240710(320-327)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3240508.3240710

Index Terms

Quick Bootstrapping of a Personalized Gaze Model from Real-Use Interactions
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Building a Self-Learning Eye Gaze Model from User Interaction Data
MM '14: Proceedings of the 22nd ACM international conference on Multimedia

Most eye gaze estimation systems rely on explicit calibration, which is inconvenient to the user, limits the amount of possible training data and consequently the performance. Since there is likely a strong correlation between gaze and interaction cues, ...
Eye-Model-Based Gaze Estimation by RGB-D Camera
CVPRW '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops

This paper proposes a method of eye-model-based gaze estimation by RGB-D camera, Kinect sensor. Different from other methods, our method sets up a model to calibrate the eyeball center by gazing at a target in 3D space, not predefined. And then by ...
A non-contact device for tracking gaze in a human computer interface
Special issue on eye detection and tracking

This paper presents a novel design for a non-contact eye detection and gaze tracking device. It uses two cameras to maintain real-time tracking of a person's eye in the presence of head motion. Image analysis techniques are used to obtain accurate ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 9, Issue 4

Research Survey and Regular Papers

July 2018

280 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3183892

Editor:
Yu Zheng
Microsoft Research, China

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 January 2018

Accepted: 01 October 2017

Revised: 01 October 2017

Received: 01 May 2017

Published in TIST Volume 9, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Hong Kong Polytechnic University
Hong Kong Research Grant Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
284
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bisogni CNappi MTortora GDel Bimbo A(2024)Gaze analysisImage and Vision Computing10.1016/j.imavis.2024.104961144:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.imavis.2024.104961
Fu EHuang MLeong HNgai GBoll SMu Lee KLuo JZhu WByun HWen Chen CLienhart RMei T(2018)Cross-Species LearningProceedings of the 26th ACM international conference on Multimedia10.1145/3240508.3240710(320-327)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3240508.3240710

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents