[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3536221.3556580acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article
Public Access

Evaluating Calibration-free Webcam-based Eye Tracking for Gaze-based User Modeling

Published: 07 November 2022 Publication History

Abstract

Eye tracking has been a research tool for decades, providing insights into interactions, usability, and, more recently, gaze-enabled interfaces. Recent work has utilized consumer-grade and webcam-based eye tracking, but is limited by the need to repeatedly calibrate the tracker, which becomes cumbersome for use outside the lab. To address this limitation, we developed an unsupervised algorithm that maps gaze vectors from a webcam to fixation features used for user modeling, bypassing the need for screen-based gaze coordinates, which require a calibration process. We evaluated our approach using three datasets (N=377) encompassing different UIs (computerized reading, an Intelligent Tutoring System), environments (laboratory or the classroom), and a traditional gaze tracker used for comparison. Our research shows that webcam-based gaze features correlate moderately with eye-tracker-based features and can model user engagement and comprehension as accurately as the latter. We discuss applications for research and gaze-enabled user interfaces for long-term use in the wild.

References

[1]
Alnajar, F., Gevers, T., Valenti, R. and Ghebreab, S. 2013. Calibration-free gaze estimation using human gaze patterns. Proceedings of the IEEE international conference on computer vision (2013), 137–144.
[2]
Bacivarov, I., Ionita, M. and Corcoran, P. 2008. Statistical Models of Appearance for Eye Tracking and Eye-Blink Detection and Measurement. IEEE Transactions on Consumer Electronics. 54, 3 (2008), 1312–1328.https://doi.org/10.1109/TCE.2008.4637622.
[3]
Bal, E., Harden, E., Lamb, D., Van Hecke, A.V., Denver, J.W. and Porges, S.W. 2010. Emotion Recognition in Children with Autism Spectrum Disorders: Relations to Eye Gaze and Autonomic State. Journal of Autism and Developmental Disorders. 40, 3 (2010), 358–370.
[4]
Baltrušaitis, T., Robinson, P. and Morency, L.P. 2013. Constrained local neural fields for robust facial landmark detection in the wild. Proceedings of the IEEE International Conference on Computer Vision (2013).https://doi.org/10.1109/ICCVW.2013.54.
[5]
Baltrusaitis, T., Zadeh, A., Lim, Y.C. and Morency, L.P. 2018. OpenFace 2.0: Facial Behavior Analysis Toolkit. Proceedings - 13th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2018. (May 2018), 59–66.https://doi.org/10.1109/FG.2018.00019.
[6]
Baluja, S. and Pomerleau, D. 1994. Non-Intrusive Gaze Tracking Using Artificial Neural Networks, CMU-CS-94-102. Neural Networks. (1994), 753–760.
[7]
Bixler, R. and D'Mello, S.K. 2016. Automatic Gaze-Based User-Independent Detection of Mind Wandering during Computerized Reading. User Modeling and User-Adapted Interaction. 26, 1 (2016), 33–68.https://doi.org/10.1007/s11257-015-9167-1.
[8]
Bixler, R. and D'Mello, S.K. 2014. Toward Fully Automated Person-Independent Detection of Mind Wandering. User Modeling Adaptation and Personalization (Aalborg, Denmark, Denmark, 2014), 37–48.https://doi.org/10.1007/978-3-319-08786-3_4.
[9]
Bosch, N. and D'Mello, S.K. 2022. Can Computers Outperform Humans in Detecting User Zone-Outs? Implications for Intelligent Interfaces. ACM Trans. Comput.-Hum. Interact. 29, 2 (Jan. 2022).https://doi.org/10.1145/3481889.
[10]
Boys, C. V 1959. Soap Bubbles, Their Colours and the Forces Which Mold Them. (1959).
[11]
Capozzi, F. and Ristic, J. 2021. Attentional Gaze Dynamics in Group Interactions. Visual Cognition. (2021), 1–16.
[12]
Cohen, P., West, S.G. and Aiken, L.S. 2014. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. (2014).
[13]
Copeland, L., Gedeon, T. and Mendis, B.S.U. 2014. Predicting Reading Comprehension Scores from Eye Movements Using Artificial Neural Networks and Fuzzy Output Error. Artif. Intell. Res. 3, 3 (2014), 35–48.
[14]
D'Mello, S.K. 2018. What Do We Think About When We Learn? Deep Comprehension. (2018), 52–67.
[15]
D'Mello, S.K., Mills, C., Bixler, R. and Bosch, N. 2017. Zone out No More: Mitigating Mind Wandering during Computerized Reading. International Educational Data Mining Society.
[16]
D'Mello, S.K., Southwell, R. and Gregg, J. 2020. Machine-Learned Computational Models Can Enhance the Study of Text and Discourse: A Case Study Using Eye Tracking to Model Reading Comprehension. Discourse Processes. (2020).https://doi.org/10.1080/0163853X.2020.1739600.
[17]
D, M.L.R. and Biswas, P. 2021. Appearance-Based Gaze Estimation Using Attention and Difference Mechanism. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (Jun. 2021), 3143–3152.
[18]
Deubel, H. and Schneider, W.X. 1996. Saccade Target Selection and Object Recognition: Evidence for a Common Attentional Mechanism. Vision Research. 36, 12 (1996), 1827–1837.https://doi.org/10.1016/0042-6989(95)00294-4.
[19]
Ester, Kriegel, Sander and Xu 1996. DBSCAN. KDD-96 Proceedings. (1996).https://doi.org/10.1.1.71.1980.
[20]
Faber, M., Bixler, R. and D'Mello, S.K. 2018. An Automated Behavioral Measure of Mind Wandering during Computerized Reading. Behavior Research Methods. 50, 1 (2018), 134–150.https://doi.org/10.3758/s13428-017-0857-y.
[21]
Franklin, M.S., Smallwood, J. and Schooler, J.W. 2011. Catching the Mind in Flight: Using Behavioral Indices to Detect Mindless Reading in Real Time. Psychonomic Bulletin & Review. 18, 5 (Oct. 2011), 992–997.https://doi.org/10.3758/s13423-011-0109-6.
[22]
Garvie, C. and Frankle, J. 2016. Facial-Recognition Software Might Have a Racial Bias Problem. The Atlantic. (2016).
[23]
Guestrin, E.D. and Eizenman, M. 2006. General Theory of Remote Gaze Estimation Using the Pupil Center and Corneal Reflections. IEEE Transactions on Biomedical Engineering. (2006).https://doi.org/10.1109/TBME.2005.863952.
[24]
Hayward, D.A., Voorhies, W., Morris, J.L., Capozzi, F. and Ristic, J. 2017. Staring Reality in the Face: A Comparison of Social Attention Across Laboratory and Real World Measures Suggests Little Common Ground. Canadian Journal of Experimental Psychology. (2017).https://doi.org/10.1037/cep0000117.
[25]
Hoffman, J.E. and Subramaniam, B. 1995. The Role of Visual Attention in Saccadic Eye Movements. Perception & Psychophysics. 57, 6 (1995), 787–795.https://doi.org/10.3758/BF03206794.
[26]
Hutt, S., Krasich, K., Brockmole, J.R. and D'Mello, S.K. 2021. Breaking Out of the Lab: Mitigating Ming Wandering with Gaze-Based Attention-Aware Technology in Classrooms. ACM SIGCHI: Computer-Human Interaction (2021).
[27]
Hutt, S., Krasich, K., Mills, C., Bosch, N., White, S., Brockmole, J.R. and D'Mello, S.K. 2019. Automated Gaze-Based Mind Wandering Detection during Computerized Learning in Classrooms. User Modeling and User-Adapted Interaction. 29, 4 (Sep. 2019), 821–867.https://doi.org/10.1007/s11257-019-09228-5.
[28]
Ince, I.F. and Kim, J.W. 2011. A 2D Eye Gaze Estimation System with Low-Resolution Webcam Images. Eurasip Journal on Advances in Signal Processing. 2011, (2011), 1–11.https://doi.org/10.1186/1687-6180-2011-40.
[29]
Itier, R.J. and Batty, M. 2009. Neural Bases of Eye and Gaze Processing: The Core of Social Cognition. Neuroscience &Biobehavioral Reviews. 33, 6 (2009), 843–863.
[30]
Just, M.A. and Carpenter, P. 1976. Eye Fixations and Cognitive Processes. Cognitive Psychology. 8, 4 (1976), 441–480.https://doi.org/10.1016/0010-0285(76)90015-3.
[31]
Kar, A. and Corcoran, P. 2017. A review and analysis of eye-gaze estimation systems, algorithms and performance evaluation methods in consumer platforms. IEEE Access.
[32]
Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H. and Hua, G. 2016. Labeled Faces in the Wild: A Survey. Advances in Face Detection and Facial Image Analysis. (2016).https://doi.org/10.1007/978-3-319-25958-1_8.
[33]
Lin, Y.T., Lin, R.Y., Lin, Y.C. and Lee, G.C. 2013. Real-Time Eye-Gaze Estimation Using a Low-Resolution Webcam. Multimedia Tools and Applications. 65, 3 (2013), 543–568.https://doi.org/10.1007/s11042-012-1202-1.
[34]
Malhotra, N.K., Kim, S.S. and Agarwal, J. 2004. Internet users’ information privacy concerns (IUIPC): The construct, the scale, and a causal model. Information Systems Research.
[35]
Mathews, A., Fox, E., Yiend, J. and Calder, A. 2003. The Face of Fear: Effects of Eye Gaze and Emotion on Visual Attention. Visual Cognition. 10, 7 (2003), 823–835.
[36]
Mills, C., D'Mello, S.K., Bosch, N. and Olney, A.M. 2015. Mind Wandering During Learning with an Intelligent Tutoring System. Artificial Intelligence in Education (Madrid, Spain, Spain, Jun. 2015), 267–276.https://doi.org/10.1007/978-3-319-19773-9_27.
[37]
Mills, C., Gregg, J.M., Bixler, R. and D'Mello, S.K. 2021. Eye-Mind Reader: An Intelligent Reading Interface That Promotes Long-Term Comprehension by Detecting and Responding to Mind Wandering. Hum. Comput. Interact. 36, 4 (2021), 306–332.https://doi.org/10.1080/07370024.2020.1716762.
[38]
Müller, P., Huang, M.X., Zhang, X. and Bulling, A. 2018. Robust eye contact detection in natural multi-person interactions using gaze and speaking behaviour. Eye Tracking Research and Applications Symposium (ETRA) (2018).https://doi.org/10.1145/3204493.3204549.
[39]
Nguyen, D.H., Bedford, A., Bretana, A.G. and Hayes, G.R. 2011. Situating the Concern for Information Privacy through an Empirical Study of Responses to Video Recording. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. (2011), 3207–3216.https://doi.org/10.1145/1978942.1979419.
[40]
Olney, A.M., D'Mello, S.K., Person, N., Cade, W., Hays, P., Williams, C., Lehman, B. and Graesser, A. 2012. Guru: A Computer Tutor That Models Expert Human Tutors. Intelligent Tutoring Systems (Chania, Crete, Greece, Jun. 2012), 256–261.https://doi.org/10.1007/978-3-642-30950-2_32.
[41]
Olney, A.M., Person, N.K. and Graesser, A.C. 2012. Guru: Designing a Conversational Expert Intelligent Tutoring System. Cross-Disciplinary Advances in Applied Natural Language Processing. (2012), 156–171.https://doi.org/10.4018/978-1-61350-447-5.ch011.
[42]
Papoutsaki, A., Laskey, J. and Huang, J. 2017. SearchGazer: Webcam Eye Tracking for Remote Studies of Web Search. Proceedings of the 2017 Conference Human Information Interaction and Retrieval (2017), 17–26.https://doi.org/10.1145/3020165.3020170.
[43]
Papoutsaki, A., Sangkloy, P., Laskey, J., Daskalova, N., Huang, J. and Hays, J. 2016. WebGazer: Scalable Webcam Eye Tracking Using User Interactions. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (2016), 3839–3845.
[44]
Paul, C., Scheibe, K. and Nilakanta, S. 2020. Privacy Concerns Regarding Wearable IoT Devices: How it is Influenced by GDPR? Proceedings of the 53rd Hawaii International Conference on System Sciences (2020), 4388–4397.https://doi.org/10.24251/hicss.2020.536.
[45]
Pedregosa, F. 2011. Scikit-Learn: Machine Learning in Python. Journal of Machine Learning Research. 12, (2011), 2825–2830.https://doi.org/10.1007/s13398-014-0173-7.2.
[46]
Ramadan, S., Abd-almageed, W. and Smith, C.E. 2002. Eye Tracking Using Active Deformable Models. December (2002).
[47]
Randall, J.G., Oswald, F.L. and Beier, M.E. 2014. Mind-Wandering, Cognition, and Performance: A Theory-Driven Meta-Analysis of Attention Regulation. Psychological Bulletin. 140, 6 (Nov. 2014), 1411–1431.https://doi.org/10.1037/a0037428.
[48]
Rayner, K. 2009. Eye Movements and Attention in Reading, Scene Perception, and Visual Search. Quarterly Journal of Experimental Psychology. 62, 8 (2009).https://doi.org/10.1080/17470210902816461.
[49]
Rayner, K. 1998. Eye Movements in Reading and Information Processing: 20 Years of Research. Psychological Bulletin. 124, 3 (Nov. 1998), 372–422.https://doi.org/10.1037/0033-2909.124.3.372.
[50]
Reinders, M.J.T. 2014. Eye Tracking by Template Matching Using an Automatic Codebook Generation Scheme. January 1997 (2014).
[51]
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C. and Müller, M. 2011. PROC: An Open-Source Package for R and S+ to Analyze and Compare ROC Curves. BMC Bioinformatics. 12, 1 (2011), 1–8.
[52]
Schubert, E., Sander, J., Ester, M., Kriegel, H.P. and Xu, X. 2017. DBSCAN Revisited, Revisited: Why and How You Should (Still) Use DBSCAN. ACM Transactions on Database Systems. (2017).https://doi.org/10.1145/3068335.
[53]
Smallwood, J. and Schooler, J.W. 2006. The Restless Mind. Psychological Bulletin. 132, 6 (Nov. 2006), 946–958.https://doi.org/10.1037/0033-2909.132.6.946.
[54]
Southwell, R., Gregg, J., Bixler, R. and D'Mello, S.K. 2020. What Eye Movements Reveal About Later Comprehension of Long Connected Texts. Cognitive Science. 44, 10 (2020), e12905.
[55]
Sugano, Y., Matsushita, Y. and Sato, Y. 2012. Appearance-Based Gaze Estimation Using Visual Saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence. 35, 2 (2012), 329–341.
[56]
Tran, M., Sen, T., Haut, K., Ali, M.R. and Hoque, M.E. 2020. Are You Really Looking at Me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze from Conventional Video. IEEE Transactions on Affective Computing. (2020), 1.https://doi.org/10.1109/TAFFC.2020.2979440.
[57]
Valliappan, N., Dai, N., Steinberg, E., He, J., Rogers, K., Ramachandran, V., Xu, P., Shojaeizadeh, M., Guo, L., Kohlhoff, K. and Navalpakkam, V. 2020. Accelerating Eye Movement Research via Accurate and Affordable Smartphone Eye Tracking. Nature Communications. 11, 1 (2020), 4553.https://doi.org/10.1038/s41467-020-18360-5.
[58]
Voßkühler, A., Nordmeier, V., Kuchinke, L. and Jacobs, A.M. 2008. OGAMA (Open Gaze and Mouse Analyzer): Open-Source Software Designed to Analyze Eye and Mouse Movements in Slideshow Study Designs. Behavior Research Methods. (2008).https://doi.org/10.3758/BRM.40.4.1150.
[59]
Wood, E., Baltruaitis, T., Zhang, X., Sugano, Y., Robinson, P. and Bulling, A. 2015. Rendering of eyes for eye-shape registration and gaze estimation. Proceedings of the IEEE International Conference on Computer Vision (2015).https://doi.org/10.1109/ICCV.2015.428.
[60]
Wu, Y.L., Yeh, C.T., Hung, W.C. and Tang, C.Y. 2014. Gaze Direction Estimation Using Support Vector Machine with Active Appearance Model. Multimedia Tools and Applications. 70, 3 (2014), 2037–2062.https://doi.org/10.1007/s11042-012-1220-z.
[61]
Yonetani, R., Kawashima, H. and Matsuyama, T. 2012. Multi-mode Saliency Dynamics Model for Analyzing Gaze and Attention. Proceedings of the Symposium on Eye Tracking Research and Applications (New York, NY, USA, 2012), 115–122.https://doi.org/10.1145/2168556.2168574.
[62]
Zhang, X., Sugano, Y. and Bulling, A. 2017. Everyday eye contact detection using unsupervised gaze target discovery. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (2017).https://doi.org/10.1145/3126594.3126614.
[63]
Zhang, X., Sugano, Y., Fritz, M. and Bulling, A. 2019. MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 41, 1 (2019), 162–175.https://doi.org/10.1109/TPAMI.2017.2778103.
[64]
Zhao, Y., Lofi, C. and Hauff, C. 2017. Scalable Mind-Wandering Detection for MOOCs: A Webcam-Based Approach. Data Driven Approaches in Digital Education - 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12-15, 2017, Proceedings (2017), 330–344.https://doi.org/10.1007/978-3-319-66610-5\_24.
[65]
Zhao, Y., Lofi, C. and Hauff, C. 2017. Scalable Mind-Wandering Detection for MOOCs: A Webcam-Based Approach. Data Driven Approaches in Digital Education - 12th European Conference on Technology Enhanced Learning, EC-TEL 2017, Tallinn, Estonia, September 12-15, 2017, Proceedings (2017), 330–344.https://doi.org/10.1007/978-3-319-66610-5\_24.

Cited By

View all
  • (2024)A Review of Key Factors Influencing Mind Wandering in Online Learning and the Role of Visual Design and Technological InterventionsIEEE Access10.1109/ACCESS.2024.350657612(177856-177895)Online publication date: 2024
  • (2024)Facial Expressions Based on the Types of Conversation ContentsThe Review of Socionetwork Strategies10.1007/s12626-024-00177-z18:2(449-489)Online publication date: 16-Nov-2024
  • (2023)Validation of an open source, remote web‐based eye‐tracking method (WebGazer) for research in early childhoodInfancy10.1111/infa.1256429:1(31-55)Online publication date: 18-Oct-2023
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '22: Proceedings of the 2022 International Conference on Multimodal Interaction
November 2022
830 pages
ISBN:9781450393904
DOI:10.1145/3536221
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Comprehension
  2. DBSCAN
  3. Eye Tracking
  4. Gaze-enabled interfaces
  5. Mind Wandering
  6. Webcam

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

ICMI '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 321 of 785 submissions, 41%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)409
  • Downloads (Last 6 weeks)53
Reflects downloads up to 12 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)A Review of Key Factors Influencing Mind Wandering in Online Learning and the Role of Visual Design and Technological InterventionsIEEE Access10.1109/ACCESS.2024.350657612(177856-177895)Online publication date: 2024
  • (2024)Facial Expressions Based on the Types of Conversation ContentsThe Review of Socionetwork Strategies10.1007/s12626-024-00177-z18:2(449-489)Online publication date: 16-Nov-2024
  • (2023)Validation of an open source, remote web‐based eye‐tracking method (WebGazer) for research in early childhoodInfancy10.1111/infa.1256429:1(31-55)Online publication date: 18-Oct-2023
  • (2023)Engagement Detection and Its Applications in Learning: A Tutorial and Selective ReviewProceedings of the IEEE10.1109/JPROC.2023.3309560111:10(1398-1422)Online publication date: Oct-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media