More Web Proxy on the site http://driver.im/

research-article

Measuring Students’ Self-Regulatory Phases in LMS with Behavior and Real-Time Self Report

Authors:

Fatemeh Salehian Kia,

Stephanie D. TeasleyAuthors Info & Claims

LAK21: LAK21: 11th International Learning Analytics and Knowledge Conference

Pages 259 - 268

https://doi.org/10.1145/3448139.3448164

Published: 12 April 2021 Publication History

Abstract

Research has emphasized that self-regulated learning (SRL) is critically important for learning. However, students have different capabilities of regulating their learning processes and individual needs. To help students improve their SRL capabilities, we need to identify students’ current behaviors. Specifically, we applied instructional design to create visible and meaningful markers of student learning at different points in time in LMS logs. We adopted knowledge engineering to develop a framework of proximal indicators representing SRL phases and evaluated them in a quasi-experiment in two different learning activities. A comparison of two sources of collected students’ SRL data, self-reported and trace data, revealed a relatively high agreement between our classifications (weighted kappa, κ = .74 and κ = .68). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting phases, compared with students’ real-time self-reported behaviors. Our behavioral indicators also were comparably successful at classifying SRL phases for different self-regulatory engagement levels. This study demonstrated how the triangulation of various sources of students’ self-regulatory data could help to unravel the complex nature of metacognitive processes.

References

[1]

Roger Azevedo and Dragan Gašević. 2019. Analyzing multimodal multichannel data about self-regulated learning with advanced learning technologies: Issues and challenges.

[2]

Roger Azevedo, Michelle Taub, Nicholas V Mudrick, SA Martin, and J Grafsgaard. 2018. Using multi-channel trace data to infer and foster self-regulated learning between humans and advanced learning technologies. Handbook of self-regulation of learning and performance 2 (2018).

[3]

Ryan SJD Baker, Albert T Corbett, and Angela Z Wagner. 2006. Human classification of low-fidelity replays of student actions. In Proceedings of the Educational Data Mining Workshop at the 8th International Conference on Intelligent Tutoring Systems, Vol. 2002. 29–36.

[4]

Albert Bandura. 1993. Perceived self-efficacy in cognitive development and functioning. Educational psychologist 28, 2 (1993), 117–148.

[5]

Maria Bannert, Peter Reimann, and Christoph Sonnenberg. 2014. Process mining techniques for analysing patterns and strategies in students’ self-regulated learning. Metacognition and learning 9, 2 (2014), 161–185.

[6]

Sanam Shirazi Beheshitha, Dragan Gašević, and Marek Hatala. 2015. A process mining approach to linking the study of aptitude and event facets of self-regulated learning. In Proceedings of the fifth international conference on learning analytics and knowledge. 265–269.

Digital Library

[7]

Deborah L Butler and Philip H Winne. 1995. Feedback and self-regulated learning: A theoretical synthesis. Review of educational research 65, 3 (1995), 245–281.

[8]

Analía Cicchinelli, Eduardo Veas, Abelardo Pardo, Viktoria Pammer-Schindler, Angela Fessl, Carla Barreiros, and Stefanie Lindstädt. 2018. Finding traces of self-regulated learning in activity streams. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge. ACM, 191–200.

Digital Library

[9]

Timothy J Cleary, Gregory L Callan, Jaime Malatesta, and Tanya Adams. 2015. Examining the level of convergence among self-regulated learning microanalytic processes, achievement, and a self-report questionnaire. Journal of Psychoeducational Assessment 33, 5 (2015), 439–450.

[10]

Jacob Cohen. 1968. Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit.Psychological bulletin 70, 4 (1968), 213–220.

[11]

Kym Craig, Daniel Hale, Catherine Grainger, and Mary E Stewart. 2020. Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning 15, 2 (2020), 155–213.

[12]

K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data.The MIT Press.

[13]

Dragan Gašević, Shane Dawson, Tim Rogers, and Danijela Gasevic. 2016. Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28 (2016), 68–84.

[14]

Dragan Gasevic, Jelena Jovanovic, Abelardo Pardo, and Shane Dawson. 2017. Detecting learning strategies with analytics: Links with self-reported measures and academic performance. Journal of Learning Analytics 4, 2 (2017), 113–128.

[15]

Jeffrey A Greene and Roger Azevedo. 2010. The measurement of learners’ self-regulated cognitive and metacognitive processes while using computer-based learning environments. Educational psychologist 45, 4 (2010), 203–209.

[16]

Jeffrey A Greene and Dale H Schunk. 2017. Historical, contemporary, and future perspectives on self-regulated learning and performance. In Handbook of Self-Regulation of Learning and Performance. Routledge, 17–32.

[17]

Allyson F Hadwin, John C Nesbit, Dianne Jamieson-Noel, Jillianne Code, and Philip H Winne. 2007. Examining trace data to explore self-regulated learning. Metacognition and Learning 2, 2-3 (2007), 107–124.

[18]

Dirk Ifenthaler. 2012. Determining the effectiveness of prompts for self-regulated learning in problem-solving scenarios. Journal of Educational Technology & Society 15, 1 (2012), 38–52.

[19]

Jelena Jovanović, Dragan Gašević, Shane Dawson, Abelardo Pardo, Negin Mirriahi, 2017. Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education 33, 4 (2017), 74–85.

[20]

Stuart A Karabenick and Akane Zusho. 2015. Examining approaches to research on self-regulated learning: conceptual and methodological considerations. Metacognition and Learning 10, 1 (2015), 151–163.

[21]

J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.

[22]

Steven Lonn, Stephen Aguilar, and Stephanie D Teasley. 2013. Issues, challenges, and lessons learned when scaling up a learning analytics intervention. In Proceedings of the third international conference on learning analytics and knowledge. ACM, 235–239.

Digital Library

[23]

Jonna Malmberg, Sanna Järvelä, Jukka Holappa, Eetu Haataja, Xiaohua Huang, and Antti Siipo. 2019. Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning?Computers in Human Behavior 96 (2019), 235–245.

[24]

Wannisa Matcha, Dragan Gašević, Jelena Jovanović, Abelardo Pardo, Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, 2019. Detection of Learning Strategies: A Comparison of Process, Sequence and Network Analytic Approaches. In European Conference on Technology Enhanced Learning. Springer, 525–540.

[25]

Nicholas V Mudrick, Roger Azevedo, and Michelle Taub. 2019. Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning. Computers in Human Behavior 96 (2019), 223–234.

Digital Library

[26]

John Perry, David Lundie, and Gill Golder. 2019. Metacognition in schools: what does the literature suggest about the effectiveness of teaching metacognition in schools?Educational Review 71, 4 (2019), 483–500.

[27]

Paul R Pintrich. 1995. Understanding self-regulated learning. New directions for teaching and learning 1995, 63 (1995), 3–12.

[28]

Paul R Pintrich and Elisabeth V De Groot. 1990. Motivational and self-regulated learning components of classroom academic performance.Journal of educational psychology 82, 1 (1990), 33–40.

[29]

Paul R Pintrich, David AF Smith, Teresa Garcia, and Wilbert J McKeachie. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and psychological measurement 53, 3 (1993), 801–813.

[30]

Bart Rienties, Lisette Toetenel, and Annie Bryan. 2015. Scaling up learning design: impact of learning design activities on lms behavior and performance. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge. ACM, 315–319.

Digital Library

[31]

Ido Roll and Philip H Winne. 2015. Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics 2, 1 (2015), 7–12.

[32]

Sanne FE Rovers, Geraldine Clarebout, Hans HCM Savelberg, Anique BH de Bruin, and Jeroen JG van Merriënboer. 2019. Granularity matters: comparing different ways of measuring self-regulated learning. Metacognition and Learning 14, 1 (2019), 1–19.

[33]

Neil J Salkind. 2006. Encyclopedia of measurement and statistics. SAGE publications.

[34]

Michael A Sao Pedro, Ryan SJD Baker, and Janice D Gobert. 2013. What different kinds of stratification can reveal about the generalizability of data-mined skill assessment models. In Proceedings of the Third International Conference on Learning Analytics and Knowledge. 190–194.

Digital Library

[35]

Benjamin Saunders, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette Bartlam, Heather Burroughs, and Clare Jinks. 2018. Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & quantity 52, 4 (2018), 1893–1907.

[36]

John R Savery. 2015. Overview of problem-based learning: Definitions and distinctions. Essential readings in problem-based learning: Exploring and extending the legacy of Howard S. Barrows 9 (2015), 5–15.

[37]

Gayane Sedrakyan, Jonna Malmberg, Katrien Verbert, Sanna Järvelä, and Paul A Kirschner. 2018. Linking learning behavior analytics and learning science concepts: designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior(2018).

[38]

James R Segedy, John S Kinnebrew, and Gautam Biswas. 2015. Coherence over time: understanding day-to-day changes in students’ open-ended problem solving behaviors. In International Conference on Artificial Intelligence in Education. Springer, 449–458.

[39]

James R Segedy, John S Kinnebrew, and Gautam Biswas. 2015. Using Coherence Analysis to Characterize Self-Regulated Learning Behaviours in Open-Ended Learning Environments.Journal of Learning Analytics 2, 1 (2015), 13–48.

[40]

Christoph Sonnenberg and Maria Bannert. 2019. Using Process Mining to examine the sustainability of instructional support: How stable are the effects of metacognitive prompting on self-regulatory behavior?Computers in Human Behavior 96 (2019), 259–272.

[41]

Philip H Winne. 2018. Cognition and metacognition within self-regulated learning.(2018).

[42]

Philip H Winne and Allyson F Hadwin. 1998. Studying as self-regulated engagement in learning. Metacognition in educational theory and practice (1998), 277–304.

[43]

Philip H Winne and Dianne Jamieson-Noel. 2002. Exploring students’ calibration of self reports about study tactics and achievement. Contemporary Educational Psychology 27, 4 (2002), 551–572.

[44]

Mingming Zhou and Philip H Winne. 2012. Modeling academic achievement by self-reported versus traced goal orientation. Learning and Instruction 22, 6 (2012), 413–419.

Cited By

Figueiredo da Silva C(2024)Abordagens didáticas e inteligência artificialIntercontinental Journal on Physical Education10.51995/2675-0333.v6i1e2020053(1-15)Online publication date: 23-Oct-2024
https://doi.org/10.51995/2675-0333.v6i1e2020053
Pansri BSharma STimilsina SChoonhapong WKurashige KWatanabe SSato K(2024)Understanding Student Learning Behavior: Integrating the Self-Regulated Learning Approach and K-Means ClusteringEducation Sciences10.3390/educsci1412129114:12(1291)Online publication date: 25-Nov-2024
https://doi.org/10.3390/educsci14121291
Jovanovic JGašević DYan LBaker GMurray AGasevic D(2024)Explaining trace‐based learner profiles with self‐reports: The added value of psychological networksJournal of Computer Assisted Learning10.1111/jcal.1296840:4(1481-1499)Online publication date: 4-Mar-2024
https://doi.org/10.1111/jcal.12968
Show More Cited By

Recommendations

Promoting self-regulated learning in web-based learning environments

Self-regulated learning with the Internet or hypermedia requires not only cognitive learning strategies, but also specific and general meta-cognitive strategies. The purposes of the Study2000 project, carried out at the TU Dresden, were to develop and ...
The Dynamic Nature of Self-Regulatory Behavior in Self-Regulated Learning and Externally-Regulated Learning Episodes
Proceedings of the 2007 conference on Artificial Intelligence in Education: Building Technology Rich Learning Contexts That Work

This study empirically examines the temporal nature of students' self-regulatory behavior while learning about a complex science topic using hypermedia. The experiment involved randomly assigning 74 undergraduate students to one of two tutoring ...
Supporting learners' self-regulated learning in Massive Open Online Courses
Abstract
In MOOCs, learners are typically presented with great autonomy over their learning process. Therefore, learners should engage in self-regulated learning (SRL) in order to successfully study in a MOOC. Learners however often struggle to self-...
Highlights
- Learners struggle to regulate their learning in massive open online courses (MOOCs).
- A self-regulated learning (SRL) intervention was implemented in three MOOCs.
- Learners' SRL was measured with trace data variables.
- ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK21: LAK21: 11th International Learning Analytics and Knowledge Conference

April 2021

645 pages

ISBN:9781450389358

DOI:10.1145/3448139

Program Chairs:
Maren Scheffel
Ruhr University Bochum, Germany
,
Nia Dowell
University of California, Irvine, USA
,
Srecko Joksimovic
University of South Australia, Australia
,
George Siemens
University of Texas, Arlington, USA & University of South Australia, Australia

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

the Natural Sciences and Engineering Research Council of Canada (NSERC); Social Sciences and Humanities Research Council (SSHRC)

Conference

LAK21

LAK21: 11th International Learning Analytics and Knowledge Conference

April 12 - 16, 2021

CA, Irvine, USA

Acceptance Rates

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
592
Total Downloads

Downloads (Last 12 months)121
Downloads (Last 6 weeks)21

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Figueiredo da Silva C(2024)Abordagens didáticas e inteligência artificialIntercontinental Journal on Physical Education10.51995/2675-0333.v6i1e2020053(1-15)Online publication date: 23-Oct-2024
https://doi.org/10.51995/2675-0333.v6i1e2020053
Pansri BSharma STimilsina SChoonhapong WKurashige KWatanabe SSato K(2024)Understanding Student Learning Behavior: Integrating the Self-Regulated Learning Approach and K-Means ClusteringEducation Sciences10.3390/educsci1412129114:12(1291)Online publication date: 25-Nov-2024
https://doi.org/10.3390/educsci14121291
Jovanovic JGašević DYan LBaker GMurray AGasevic D(2024)Explaining trace‐based learner profiles with self‐reports: The added value of psychological networksJournal of Computer Assisted Learning10.1111/jcal.1296840:4(1481-1499)Online publication date: 4-Mar-2024
https://doi.org/10.1111/jcal.12968
Çakiroğlu ÜKokoç MAtabay M(2024)Online learners’ self-regulated learning skills regarding LMS interactions: a profiling studyJournal of Computing in Higher Education10.1007/s12528-024-09397-236:1(220-241)Online publication date: 16-Feb-2024
https://doi.org/10.1007/s12528-024-09397-2
Bernacki M(2023)Development, Sustainment, and Scaling of Self-Regulated Learning AnalyticsSupporting Self-Regulated Learning and Student Success in Online Courses10.4018/978-1-6684-6500-4.ch012(255-281)Online publication date: 24-Feb-2023
https://doi.org/10.4018/978-1-6684-6500-4.ch012
Aguilar S(2023)Using Motivation Theory to Design Equity-Focused Learning Analytics DashboardsTrends in Higher Education10.3390/higheredu20200152:2(283-290)Online publication date: 29-Mar-2023
https://doi.org/10.3390/higheredu2020015
Giovanelli ARowe JTaylor MBerna MTebb KPenilla CPugatch MLester JOzer E(2023)Supporting Adolescent Engagement with Artificial Intelligence–Driven Digital Health Behavior Change InterventionsJournal of Medical Internet Research10.2196/4030625(e40306)Online publication date: 24-May-2023
https://doi.org/10.2196/40306
Berger J(2023) Motivational Beliefs, Metacognition, and Self-Regulated Learning: Investigating the Learning Triumvirate with Stuart Karabenick 1 Remembering the Life, Work, and Influence of Stuart A. Karabenick10.1108/S0749-742320230000022014(195-213)Online publication date: 23-Feb-2023
https://doi.org/10.1108/S0749-742320230000022014
Hatala MNazeri SSalehian Kia F(2023)Progression of students' SRL processes in subsequent programming problem-solving tasks and its association with tasks outcomesThe Internet and Higher Education10.1016/j.iheduc.2022.10088156(100881)Online publication date: Jan-2023
https://doi.org/10.1016/j.iheduc.2022.100881
Salehian Kia FBernacki MGreene J(2023)Measuring and Validating Assumptions About Self-Regulated Learning with Multimodal DataUnobtrusive Observations of Learning in Digital Environments10.1007/978-3-031-30992-2_9(123-140)Online publication date: 14-Jun-2023
https://doi.org/10.1007/978-3-031-30992-2_9

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents