Iterative Improvement of Automatically Generated Practice with the Content Improvement Service
Authors: 
Bill Jerome, Rachel Van Campenhout, Jeffrey S. Dittel, Rich Benton, 
Benny G. JohnsonAuthors Info & Claims
Pages 312 - 324
Abstract
The Content Improvement Service (CIS) is a platform-level adaptive system that monitors millions of automatically generated formative practice questions that are available to students as a study feature in thousands of textbooks. The CIS was designed to do what was not possible by humans—use real-time data to monitor question performance at enormous scale and determine if a change is required. The CIS was designed to use multiple types of data and analyses to make its decisions, including question difficulty (mean score) and student feedback (ratings). In this paper, we outline the decisions made by the CIS for both methods. We also show how human investigation of these analyses can identify trends and insights useful to automatic question generation systems.
References
[1]
Koedinger, K., McLaughlin, E., Jia, J., Bier, N.: Is the doer effect a causal relationship? How can we tell and why it’s important. In: Learning Analytics and Knowledge. Edinburgh, United Kingdom (2016).
[2]
Van Campenhout, R., Johnson, B.G., Olsen, J.A.: The doer effect: replication and comparison of correlational and causal analyses of learning. Int. J. Adv. Syst. Meas. 15(1&2) (2022). http://www.iariajournals.org/systems_and_measurements/tocv15n12.html
[3]
Van Campenhout, R., Jerome, B., Johnson, B.G.: The doer effect at scale: investigating correlation and causation across seven courses. In: 13th International Learning Analytics and Knowledge Conference (LAK 2023) (2023).
[4]
Jerome, B., Van Campenhout, R., Dittel, J.S., Benton, R., Greenberg, S., Johnson, B.G.: The content improvement service: an adaptive system for continuous improvement at scale. In: Meiselwitz, G., et al. (eds.) Interaction in New Media, Learning and Games (HCII 2022). LNCS, vol. 13517, pp. 286–296. Springer, Cham (2022).
[5]
Vandewaetere M, Desmet P, and Clarebout G The contribution of learner characteristics in the development of computer-based adaptive learning environments Comput. Human Behav. 2011 27 1 118-130
[6]
Simon HA The job of a college president Educ. Rec. 1967 48 68-78
[7]
Goodell, J.: What is learning engineering? In: Goodell, J., Kolodner, J. (eds.) Learning Engineering Toolkit: Evidence-Based Practices from the Learning Sciences, Instructional Design, and Beyond. Routledge, New York (2022)
[8]
Campenhout R Sottilare RA and Schwarz J Learning engineering as an ethical framework Adaptive Instructional Systems. Design and Evaluation 2021 Cham Springer 105-119
[9]
Van Campenhout, R., Dittel, J.S., Jerome, B., Johnson, B.G.: Transforming textbooks into learning by doing environments: an evaluation of textbook-based automatic question generation. In: Third Workshop on Intelligent Textbooks at the 22nd International Conference on Artificial Intelligence in Education. CEUR Workshop Proceedings, pp. 60–73 (2021). http://ceur-ws.org/Vol-2895/paper06.pdf
[10]
Kurdi G, Leo J, Parsia B, Sattler U, and Al-Emari S A systematic review of automatic question generation for educational purposes Int. J. Artif. Intell. Educ. 2019 30 1 121-204
[11]
Fischer C et al. Mining big data in education: affordances and challenges Rev. Res. Educ. 2020 44 1 130-160
[12]
Chandler P and Sweller J Cognitive load theory and the format of instruction Cogn. Instr. 1991 8 4 293-332
[13]
Sweller J Cognitive load theory and educational technology Educ. Tech. Res. Dev. 2019 68 1 1-16
Recommendations
The Content Improvement Service: An Adaptive System for Continuous Improvement at Scale
HCI International 2022 - Late Breaking Papers. Interaction in New Media, Learning and GamesAbstractAdvances in artificial intelligence and automatic question generation have made it possible to create millions of questions to apply an evidence-based learn by doing method to thousands of e-textbooks, an unprecedented scale. Yet the scaling of ...
Discrimination of Automatically Generated Questions Used as Formative Practice
L@S '22: Proceedings of the Ninth ACM Conference on Learning @ ScaleAdvances in artificial intelligence and automatic question generation (AQG) have made it possible to generate the volume of formative practice questions needed to engage students in learning by doing. These automatically generated (AG) questions can be ...
Toward Effective Courseware at Scale: Investigating Automatically Generated Questions as Formative Practice
L@S '21: Proceedings of the Eighth ACM Conference on Learning @ ScaleCourseware is a comprehensive learning environment that engages students in a learning by doing approach while also giving instructors data-driven insights on their class, providing a scalable solution for many instructional models. However, courseware-...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Jul 2023
364 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 23 July 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Dec 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in