More Web Proxy on the site http://driver.im/

Article

Modeling and understanding students' off-task behavior in intelligent tutoring systems

Author:

Ryan S.J.d. BakerAuthors Info & Claims

CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 1059 - 1068

https://doi.org/10.1145/1240624.1240785

Published: 29 April 2007 Publication History

Abstract

We present a machine-learned model that can automatically detect when a student using an intelligent tutoring system is off-task, i.e., engaged in behavior which does not involve the system or a learning task. This model was developed using only log files of system usage (i.e. no screen capture or audio/video data). We show that this model can both accurately identify each student's prevalence of off-task behavior and can distinguish off-task behavior from when the student is talking to the teacher or another student about the subject matter. We use this model in combination with motivational and attitudinal instruments, developing a profile of the attitudes and motivations associated with off-task behavior, and compare this profile to the attitudes and motivations associated with other behaviors in intelligent tutoring systems. We discuss how the model of off-task behavior can be used within interactive learning environments which respond to when students are off-task.

References

[1]

Aleven, V., McLaren, B.M., Roll, I., and Koedinger, K.R. Toward tutoring help seeking: Applying cognitive modeling to meta-cognitive skills. Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS 2004), 227--239.

[2]

Anderson, J.R., Corbett, A.T., Koedinger, K.R., Pelletier, R. Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences 4, 2 (1995), 167--207.

[3]

Amershi, S., and Conati, C. Automatic Recognition of Learner Groups in Exploratory Learning Environments. Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS 2006), 463--472.

Digital Library

[4]

Baker, R.S. (2005) Designing Intelligent Tutors That Adapt to When Students Game the System. Doctoral Dissertation. Carnegie Mellon University Technical Report CMU-HCII-05-104.

[5]

Baker, R.S., Corbett, A.T., and Koedinger, K.R. Detecting Student Misuse of Intelligent Tutoring Systems. Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS 2004), 531--540.

[6]

Baker, R.S.J.d., Corbett, A.T., Roll, I., Wagner, A.Z., and Koedinger, K.R. The Relationship Between Gaming the System and Learning in Cognitive Tutor Classrooms. Manuscript Under Review.

[7]

Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z. Off--Task Behavior in the Cognitive Tutor Classroom: When Students "Game the System". Proceedings of ACM CHI 2004: Computer-Human Interaction, 383--390.

Digital Library

[8]

Baker, R.S., Roll, I., Corbett, A.T., Koedinger, K.R. Do Performance Goals Lead Students to Game the System? Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005), 57--64.

Digital Library

[9]

Beck, J.E. Engagement tracing: using response times to model student disengagement. Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005), 88--95.

Digital Library

[10]

Bickmore, T.W., Picard, R.W. Towards Caring Machines. CHI Extended Abstracts (2004), 1489--1492.

Digital Library

[11]

Boyd, S., and Vandenberghe, L. Convex Optimization. Cambridge University Press, Cambridge, UK, 2004.

[12]

Carroll, J. A Model For School Learning. Teachers College Record 64 (1963), 723--733.

[13]

Cohen, J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1 (1960), 37--46.

[14]

Collins, R.T., Lipton, A.J., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., Wixson, L. Carnegie Mellon University Technical Report CMU-RI-TR-00-12: A System for Video Surveillance and Monitoring (2000).

[15]

Dalton, T., Martella, R.C., and Marchand-Martella, N.E. The effects of a self-management program in reducing off-task behavior. Journal of Behavioral Education 9, 3--4 (1999), 157--176.

[16]

de Vicente, A., and Pain H. Informing the detection of the students' motivational state: an empirical study. Proceedings of the 6th International Conference on Intelligent Tutoring Systems (ITS 2002), 933--943.

Digital Library

[17]

Dillon, T.W., Garner, M., Kuilboer, J., Quinn, J.D. Accounting Student Acceptance of Tax Preparation Software. Journal of Accounting and Computers 13 (1998), 17--29.

[18]

D'Mello, S.K., Craig, S.D., Gholson, B., Franklin, S., Picard, R.W., and Graesser, A.C. Integrating Affect Sensors in an Intelligent Tutoring System. Affective Interactions: The Computer in the Affective Loop Workshop at 2005 International Conference on Intelligent User Interfaces,7--13.

Digital Library

[19]

Frantom, C.G., Green, K.E., Hoffman, E.R. Measure Development: The Children's Attitudes Towards Technology Scale (CATS). Journal of Educational Computing Research 26, 3 (2002), 249--263.

[20]

Harnisch, D.L., Hill, K.T., Fyans, L.J. Development of a Shorter, More Reliable, and More Valid Measure of Test Motivation. Paper presented at the 1980 annual meeting of the National Council on Measurement in Education. ERIC Document #ED193273.

[21]

Knezek, G., Christensen, R. Computer Attitudes Questionnaire (1995). Denton, TX: Texas Center for Educational Technology.

[22]

Maris, E. Psychometric Latent Response Models. Psychometrika 60, 4 (1995), 523--547.

[23]

Mueller, C.M., and Dweck, C.S. Praise for Intelligence Can Undermine Children's Motivation and Performance. Journal of Personality and Social Psychology 75, 1 (1998), 33--52.

[24]

Murray, R.C., and vanLehn, K. Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help. Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005), 887--889.

Digital Library

[25]

Parker, G., Hadzi-Pavlovic, D. A Question of Style: Refining the Dimensions of Personality Disorder Style. Journal of Personality Disorders, 15, 4 (2001), 300--318.

[26]

Ramsey, F.L., Schafer, D.W. The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press, Belmont, CA, USA, 1997.

[27]

Sarason, S.B. Anxiety in Elementary School Children: A Report of Research. Greenwood Press, Westport, CT, USA, 1978.

[28]

Selwyn, N. Students' Attitudes Towards Computers: Validation of a Computer Attitude Scale for 16-19 Education. Computers & Education, 28 (1997), 35--41.

Digital Library

[29]

Suchman, L. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press, Cambridge, UK, 1987.

Digital Library

[30]

Yu, L., and Liu, H. Feature selection for high-dimensional data: a fast correlation-based filter solution. Proceedings of the International Conference on Machine Learning (2003), 856--863.

[31]

Ziemek, T.R. Two-D or not Two-D. Gender Implications of Visual Cognition in Electronic Games. Proceedings of the 2006 Symposium on Interactive 3D Graphics and Games, 183--190.

Digital Library

Cited By

Zhang LLin JSabatini JBorchers CWeitekamp DCao MHollander JHu XGraesser A(2025)Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AIIEEE Transactions on Learning Technologies10.1109/TLT.2025.352658218(145-164)Online publication date: 2025
https://doi.org/10.1109/TLT.2025.3526582
Papadimitriou SVirvou MPapadimitriou SVirvou M(2025)User-Player and Student Modeling in Personalized Educational Games: A Literature ReviewArtificial Intelligence—Based Games as Novel Holistic Educational Environments to Teach 21st Century Skills10.1007/978-3-031-77464-5_3(63-94)Online publication date: 21-Jan-2025
https://doi.org/10.1007/978-3-031-77464-5_3
Morais FGoldoni DKautzmann TJaques P(2024)Sensor-free Affect Detection in Learning Environments: A Systematic Literature ReviewRevista Brasileira de Informática na Educação10.5753/rbie.2024.436232(679-717)Online publication date: 21-Nov-2024
https://doi.org/10.5753/rbie.2024.4362
Show More Cited By

Index Terms

Modeling and understanding students' off-task behavior in intelligent tutoring systems

Recommendations

Off-task behavior in the cognitive tutor classroom: when students "game the system"
CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We investigate the prevalence and learning impact of different types of off-task behavior in classrooms where students are using intelligent tutoring software. We find that within the classrooms studied, no other type of off-task behavior is associated ...
A problem solving oriented intelligent tutoring system to improve students' acquisition of basic computer skills

Personalization and intelligent tutor are two key factors in the research on learning environment. Intelligent tutoring system (ITS), which can imitate the human teachers' actions to implement one-to-one personalized teaching to some extent, is an ...
Utilizing the Generalized Intelligent Framework for Tutoring to Encourage Self-Regulated Learning
8th International Conference on Foundations of Augmented Cognition. Advancing Human Performance and Decision-Making through Adaptive Systems - Volume 8534

Self-regulated learning is of particular importance to computer-based and online instruction, as students need to manage their own time and their interactions with the system. Elements of self-regulatory learning traditionally include the metacognitive ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2007

1654 pages

ISBN:9781595935939

DOI:10.1145/1240624

General Chair:
Mary Beth Rosson
The Pennsylvania State University, USA
,
Program Chair:
David Gilmore
Intel, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI07

Sponsor:

CHI07: CHI Conference on Human Factors in Computing Systems

April 28 - May 3, 2007

California, San Jose, USA

Acceptance Rates

CHI '07 Paper Acceptance Rate 182 of 840 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

110
Total Citations
View Citations
2,015
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)11

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang LLin JSabatini JBorchers CWeitekamp DCao MHollander JHu XGraesser A(2025)Data Augmentation for Sparse Multidimensional Learning Performance Data Using Generative AIIEEE Transactions on Learning Technologies10.1109/TLT.2025.352658218(145-164)Online publication date: 2025
https://doi.org/10.1109/TLT.2025.3526582
Papadimitriou SVirvou MPapadimitriou SVirvou M(2025)User-Player and Student Modeling in Personalized Educational Games: A Literature ReviewArtificial Intelligence—Based Games as Novel Holistic Educational Environments to Teach 21st Century Skills10.1007/978-3-031-77464-5_3(63-94)Online publication date: 21-Jan-2025
https://doi.org/10.1007/978-3-031-77464-5_3
Morais FGoldoni DKautzmann TJaques P(2024)Sensor-free Affect Detection in Learning Environments: A Systematic Literature ReviewRevista Brasileira de Informática na Educação10.5753/rbie.2024.436232(679-717)Online publication date: 21-Nov-2024
https://doi.org/10.5753/rbie.2024.4362
Hernández-Calderón JBenítez-Guerrero ERojano-Cáceres JMezura-Godoy C(2024)Mining User-Object Interaction Data for Student Modeling in Intelligent Learning EnvironmentsProgramming and Computer Software10.1134/S036176882308008X49:8(657-670)Online publication date: 24-Jan-2024
https://doi.org/10.1134/S036176882308008X
Wei YJiang B(2024)Interpretable Cognitive State Prediction via Temporal Fuzzy Cognitive MapIEEE Transactions on Learning Technologies10.1109/TLT.2023.330756517(514-526)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2023.3307565
Toutkoushian ERyoo K(2024)Modeling Complex Data from Simulations to Assess Middle School Students’ NGSS-Aligned Science LearningMeasurement: Interdisciplinary Research and Perspectives10.1080/15366367.2023.224675422:2(200-218)Online publication date: 29-Feb-2024
https://doi.org/10.1080/15366367.2023.2246754
Hershkovitz ANoster NSiller HTabach M(2024)Learning analytics in mathematics education: the case of feedback use in a digital classification task on reflective symmetryZDM – Mathematics Education10.1007/s11858-024-01551-556:4(727-739)Online publication date: 3-Mar-2024
https://doi.org/10.1007/s11858-024-01551-5
Zhang LYeasin MLin JHavugimana FHu X(2024)Generative Adversarial Networks for Imputing Sparse Learning PerformancePattern Recognition10.1007/978-3-031-78172-8_25(381-396)Online publication date: 3-Dec-2024
https://doi.org/10.1007/978-3-031-78172-8_25
Liu XGurung ABaker RBarany A(2024)Understanding the Impact of Observer Effects on Student AffectAdvances in Quantitative Ethnography10.1007/978-3-031-76332-8_7(79-94)Online publication date: 2-Nov-2024
https://doi.org/10.1007/978-3-031-76332-8_7
Yu HAllessio DRebelsky WMurray TMagee JArroyo IWoolf BBargal SBetke M(2024)Affect Behavior Prediction: Using Transformers and Timing Information to Make Early Predictions of Student Exercise OutcomeArtificial Intelligence in Education10.1007/978-3-031-64299-9_14(194-208)Online publication date: 2-Jul-2024
https://doi.org/10.1007/978-3-031-64299-9_14
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten