research-article

The effects of adaptive learning in a massive open online course on learners' skill development

Authors:

Dustin TingleyAuthors Info & Claims

L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale

Article No.: 6, Pages 1 - 8

https://doi.org/10.1145/3231644.3231651

Published: 26 June 2018 Publication History

Get Access

Abstract

We report an experimental implementation of adaptive learning functionality in a self-paced Microsoft MOOC (massive open online course) on edX. In a personalized adaptive system, the learner's progress toward clearly defined goals is continually assessed, the assessment occurs when a student is ready to demonstrate competency, and supporting materials are tailored to the needs of each learner. Despite the promise of adaptive personalized learning, there is a lack of evidence-based instructional design, transparency in many of the models and algorithms used to provide adaptive technology or a framework for rapid experimentation with different models. ALOSI (Adaptive Learning Open Source Initiative) provides open source adaptive learning technology and a common framework to measure learning gains and learner behavior. This study explored the effects of two different strategies for adaptive learning and assessment: Learners were randomly assigned to three groups. In the first adaptive group ALOSI prioritized a strategy of remediation - serving learners items on topics with the least evidence of mastery; in the second adaptive group ALOSI prioritized a strategy of continuity - that is learners would be more likely served items on similar topic in a sequence until mastery is demonstrated. The control group followed the pathways of the course as set out by the instructional designer, with no adaptive algorithms. We found that the implemented adaptivity in assessment, with emphasis on remediation is associated with a substantial increase in learning gains, while producing no big effect on the drop-out. Further research is needed to confirm these findings and explore additional possible effects and implications to course design.

References

[1]

Beck, J., Chang, K-M., Mostow, J., and Corbett, A. 2008. Does help help? Introducing the Bayesian Evaluation and Assessment methodology. In International Conference on Intelligent Tutoring Systems. Springer, 383--394

Digital Library

Google Scholar

[2]

Hawkins, W.J., Heffernan, N.T. and Baker, R.S., 2014. Learning Bayesian knowledge tracing parameters with a knowledge heuristic and empirical probabilities. In International Conference on Intelligent Tutoring Systems (pp. 150--155). Springer, Cham.

Digital Library

Google Scholar

[3]

Koedinger, K., Anderson, J., Hadley, W., and Mark, M. 1997. Intelligent tutoring goes to school in the big city. (1997).

Google Scholar

[4]

Koedinger, K., and Stamper, J. 2010. A Data Driven Approach to the Discovery of Better Cognitive Models. In Baker, R.S.J.d., Merceron, A., Pavlik, P.I. Jr. (Eds.) Proceedings of the 3rd International Conference on Educational Data Mining. (EDM 2010), 325--326. Pittsburgh, PA.

Google Scholar

[5]

Pardos, Z., and Heffernan, N. 2010. Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm. In Educational Data Mining 2010.

Google Scholar

[6]

Pardos, Z., and Heffernan, N. 2011. KT-IDEM: Introducing item difficulty to the knowledge tracing model. In International Conference on User Modeling, Adaptation, and Personalization. Springer, 243--254.

Digital Library

Google Scholar

[7]

Pardos, Z., Tang, S., Davis, D., and Vu Le C. 2017. Enabling real-time adaptivity in MOOCs with a personalized next-step recommendation framework. Proceedings of the Fourth ACM Conference on Learning @ Scale.

Digital Library

Google Scholar

[8]

Rosen, Y., Rushkin, I., Ang, A., Federicks, C., Tingley, D., and Blink, M. - J. 2017. Designing adaptive assessments in MOOCs. Proceedings of the Fourth ACM Conference on Learning @ Scale.

Digital Library

Google Scholar

[9]

Rushkin, I., Rosen, Y., Ang, A., Fredericks, C., Tingley, D., Blink, M. J., and Lopez, G. 2017. Adaptive Assessment Experiment in a HarvardX MOOC. Proceedings of the 10th International Conference on Educational Data Mining.

Google Scholar

[10]

Stamper, J., Barnes, T., and Croy, M. 2011. Experimental Evaluation of Automatic Hint Generation for a Logic Tutor. In Kay, J., Bull, S. and Biswas, G. eds. Proceeding of the 15th International Conference on Artificial Intelligence in Education (AIED2011). 345--352. Berlin Germany: Springer.

Digital Library

Google Scholar

[11]

Corbett, A. T. and Anderson, J. R., 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4), 253--278.

Google Scholar

Cited By

View all

Halkiopoulos CGkintoni E(2024)Leveraging AI in E-Learning: Personalized Learning and Adaptive Assessment through Cognitive Neuropsychology—A Systematic AnalysisElectronics10.3390/electronics1318376213:18(3762)Online publication date: 22-Sep-2024
https://doi.org/10.3390/electronics13183762
Hasanov ALaine TKim JChung T(2024)Lightlore: An Adaptation Framework for Design and Development of xAPI-Based Adaptive Context-Aware Learning EnvironmentsElectronics10.3390/electronics1313249813:13(2498)Online publication date: 26-Jun-2024
https://doi.org/10.3390/electronics13132498
du Plooy ECasteleijn DFranzsen D(2024)Personalized adaptive learning in higher education: a scoping review of key characteristics and impact on academic performance and engagementHeliyon10.1016/j.heliyon.2024.e39630(e39630)Online publication date: Oct-2024
https://doi.org/10.1016/j.heliyon.2024.e39630
Show More Cited By

Recommendations

Supporting learners' self-regulated learning in Massive Open Online Courses
Abstract
In MOOCs, learners are typically presented with great autonomy over their learning process. Therefore, learners should engage in self-regulated learning (SRL) in order to successfully study in a MOOC. Learners however often struggle to self-...
Highlights
- Learners struggle to regulate their learning in massive open online courses (MOOCs).
- A self-regulated learning (SRL) intervention was implemented in three MOOCs.
- Learners' SRL was measured with trace data variables.
- ...
Enabling Real-Time Adaptivity in MOOCs with a Personalized Next-Step Recommendation Framework
L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale

In this paper, we demonstrate a first-of-its-kind adaptive intervention in a MOOC utilizing real-time clickstream data and a novel machine learned model of behavior. We detail how we augmented the edX platform with the capabilities necessary to support ...
Investigating Learners’ Views of Assessment Types in Massive Open Online Courses (MOOCs)
Design for Teaching and Learning in a Networked World
Abstract
Massive Open Online Courses (MOOCs) are changing the contours of the teaching and learning landscape. Assessment covers an important part of this landscape and may be a key driver for learning. This paper presents preliminary results of a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale

June 2018

391 pages

ISBN:9781450358866

DOI:10.1145/3231644

Conference Chair:
Rose Luckin
UCL Institute of Education, UK
,
Program Chairs:
Scott Klemmer
University of California at San Diego
,
Kenneth Koedinger
Carnegie Mellon University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

L@S '18

L@S '18: Fifth (2018) ACM Conference on Learning @ Scale

June 26 - 28, 2018

London, United Kingdom

Acceptance Rates

L@S '18 Paper Acceptance Rate 24 of 58 submissions, 41%;

Overall Acceptance Rate 117 of 440 submissions, 27%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
543
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Halkiopoulos CGkintoni E(2024)Leveraging AI in E-Learning: Personalized Learning and Adaptive Assessment through Cognitive Neuropsychology—A Systematic AnalysisElectronics10.3390/electronics1318376213:18(3762)Online publication date: 22-Sep-2024
https://doi.org/10.3390/electronics13183762
Hasanov ALaine TKim JChung T(2024)Lightlore: An Adaptation Framework for Design and Development of xAPI-Based Adaptive Context-Aware Learning EnvironmentsElectronics10.3390/electronics1313249813:13(2498)Online publication date: 26-Jun-2024
https://doi.org/10.3390/electronics13132498
du Plooy ECasteleijn DFranzsen D(2024)Personalized adaptive learning in higher education: a scoping review of key characteristics and impact on academic performance and engagementHeliyon10.1016/j.heliyon.2024.e39630(e39630)Online publication date: Oct-2024
https://doi.org/10.1016/j.heliyon.2024.e39630
Fadieieva L(2023)Adaptive learning: a cluster-based literature review (2011-2022)Educational Technology Quarterly10.55056/etq.6132023:3(319-366)Online publication date: 12-Sep-2023
https://doi.org/10.55056/etq.613
Pardos ZTang MAnastasopoulos ISheel SZhang E(2023)OATutor: An Open-source Adaptive Tutoring System and Curated Content Library for Learning Sciences ResearchProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581574(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581574
Tazi KAzzouzi SCharaf M(2023)An Ontology-based Recommender System for Identifying Learners’ Confusion in MOOCs2023 7th IEEE Congress on Information Science and Technology (CiSt)10.1109/CiSt56084.2023.10409939(492-496)Online publication date: 16-Dec-2023
https://doi.org/10.1109/CiSt56084.2023.10409939
Ram IHarris SRoll I(2023)Choice-based Personalization in MOOCs: Impact on Activity and Perceived ValueInternational Journal of Artificial Intelligence in Education10.1007/s40593-023-00334-534:2(376-394)Online publication date: 10-Apr-2023
https://doi.org/10.1007/s40593-023-00334-5
Thompson KFarr ASaunders TWinter G(2023)The Role of Adaptive Learning Technologies and Conditional LearningTechnology-Enhanced Learning and the Virtual University10.1007/978-981-99-4170-4_26(501-523)Online publication date: 21-Sep-2023
https://doi.org/10.1007/978-981-99-4170-4_26
Wang MLi G(2022)A Meta-Analysis of MOOC-Based Academic Achievement, Engagement, Motivation, and Self-Regulation During the COVID-19 PandemicInternational Journal of e-Collaboration10.4018/IJeC.30126018:1(1-17)Online publication date: 1-Jan-2022
https://doi.org/10.4018/IJeC.301260
Guo SSun TGong JLu ZZhang LWang Q(2022)Remote Co-teaching in Rural Classroom: Current Practices, Impacts, and ChallengesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501924(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3501924
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Supporting learners' self-regulated learning in Massive Open Online Courses

Enabling Real-Time Adaptivity in MOOCs with a Personalized Next-Step Recommendation Framework

Investigating Learners’ Views of Assessment Types in Massive Open Online Courses (MOOCs)