Abstract
While learning new subjects, students often develop misconceptions that affect their results and even their academic success. Concept inventories (CIs) – collections of multiple-choice questions – are widely used instruments for spotting misconceptions and allowing instructors to correct them promptly. While the main advantage of CIs is responsiveness in analyzing answers, their measurement speed (the number of misconceptions that can be tested in a time frame) is poor. In this paper, we introduce an improved CI methodology that allows for accurate detections of broader sets of misconceptions in classes and, thus, to obtain detailed pictures of the student difficulties. Our methodology is based on observing the answers to the individual options of the multiple-choice questions, rather than to the question as a whole. We are therefore able to get more information from an item and thus improve the measurement speed. We integrated our methodology in a CS1 Java Programming Language course, and we tested 89 distinct misconceptions in 15 sessions of 30 min. Our methodology showed a 4x speed up in the measurement speed compared to state-of-the-art CIs, while preserving satisfactory accuracy. Thanks to this extensive coverage of misconceptions, we built a new metric, the “knowledge fitness”, to objectively measure student difficulties.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Science, technology, engineering, and mathematics.
- 2.
https://progmiscon.org/misconceptions/Java/AddMemberAtRuntime/.
- 3.
The participation was not mandatory, some students attended fewer sessions.
- 4.
Similarly to software instrumentation.
- 5.
Sect. 2.5.
- 6.
- 7.
That is, at the best of one’s abilities: the submission is complete and with “reasonable” explanations.
- 8.
Therefore, 2 cases for each option.
- 9.
Some of the presented figures are screenshots taken from the AMiCI - Platform.
References
Almstrum, V.L., et al.: Concept inventories in computer science for the topic discrete mathematics. SIGCSE Bull. 38(4), 132–145 (2006). https://doi.org/10.1145/1189136.1189182
Caceffo, R., Frank-Bolton, P., Souza, R., Azevedo, R.: Identifying and validating java misconceptions toward a CS1 concept inventory. In: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education, pp. 23–29. ITiCSE 2019, Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3304221.3319771
Chiodini, L., Moreno Santos, I., Gallidabino, A., Tafliovich, A., Santos, A.L., Hauswirth, M.: A curated inventory of programming language misconceptions. In: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education, vol. 1, pp. 380–386. ITiCSE 2021, Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3430665.3456343
Farghally, M.F., Koh, K.H., Ernst, J.V., Shaffer, C.A.: Towards a concept inventory for algorithm analysis topics. In: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE 2017), pp. 207–212. ACM, New York (2017). https://doi.org/10.1145/3017680.3017756
Frost, J.: Interpreting correaltion coefficients (2022). https://statisticsbyjim.com/basics/correlations/. Accessed 10 Apr 2022
Hambleton, R., Swaminathan, H.: Item Response Theory: Principles and Applications. Evaluation in education and human services, Springer, Netherlands (1985). https://books.google.ch/books?id=jKFMUFI-e1UChttps://doi.org/10.1007/978-94-017-1988-9
Hauswirth, M., Adamoli, A.: Identifying misconceptions with active recall in a blended learning system. In: Lavoué, É., Drachsler, H., Verbert, K., Broisin, J., Pérez-Sanagustín, M. (eds.) Data Driven Approaches in Digital Education, pp. 416–421. Springer International Publishing, Cham (2017)
Herman, G.L., Loui, M.C., Zilles, C.: Creating the digital logic concept inventory. In: Proceedings of the 41st ACM Technical Symposium on Computer Science Education (SIGCSE 2010), pp. 102–106. ACM, New York (2010). https://doi.org/10.1145/1734263.1734298
Hestenes, D., Wells, M., Swackhamer, G.: Force concept inventory. Phys. Teach. 30(3), 141–158 (1992). https://doi.org/10.1119/1.2343497
Kaczmarczyk, L.C., Petrick, E.R., East, J.P., Herman, G.L.: Identifying student misconceptions of programming. In: Proceedings of the 41st ACM Technical Symposium on Computer Science Education - SIGCSE 2010, p. 107. ACM Press, Milwaukee, Wisconsin (2010). https://doi.org/10.1145/1734263.1734299, http://portal.acm.org/citation.cfm?doid=1734263.1734299
Karpierz, K., Wolfman, S.A.: Misconceptions and concept inventory questions for binary search trees and hash tables. In: Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE 2014), pp. 109–114. ACM, New York (2014). https://doi.org/10.1145/2538862.2538902
Kurvinen, E., Hellgren, N., Kaila, E., Laakso, M.J., Salakoski, T.: Programming misconceptions in an introductory level programming course exam. In: Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, pp. 308–313. ITiCSE 2016, Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2899415.2899447
Loughran, J., Berry, A., Mulhall, P.: Pedagogical content knowledge. In: Understanding and Developing Science Teachers’ Pedagogical Content Knowledge, pp. 7–14. SensePublishers, Rotterdam (2012). https://doi.org/10.1007/978-94-6091-821-6_2
Porter, L., Taylor, C., Webb, K.C.: Leveraging open source principles for flexible concept inventory development. In: Proceedings of the 2014 Conference on Innovation and Technology in Computer Science Education (ITiCSE 2014), pp. 243–248. ACM, New York (2014). https://doi.org/10.1145/2591708.2591722
Qian, Y., Lehman, J.: Students’ misconceptions and other difficulties in introductory programming: a literature review. ACM Trans. Comput. Educ. 18(1), 1–24 (2017). https://doi.org/10.1145/3077618
Richardson, J.: Concept Inventories: Tools for uncovering STEM students’ misconceptions. Invention and Impact: Building Excellence in Undergraduate Science. Technology, Engineering and Mathematics (STEM) Education (2004). https://www.aaas.org/sites/default/files/02_AER_Richardson.pdf
Sadler, P.M., Sonnert, G.: Understanding misconceptions: teaching and learning in middle school physical science. Am. Educ. 40(1), 26–32 (2016)
Sherman, A.T., et al.: Experiences and lessons learned creating and validating concept inventories for cybersecurity. In: Choo, K.K.R., Morris, T., Peterson, G.L., Imsand, E. (eds.) National Cyber Summit (NCS) Research Track 2020, pp. 3–34. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-58703-1_1
Sorva, J.: Visual program simulation in introductory programming education, Ph.D. thesis, Aalto University, Espoo (2012), 00114
Acknowledgments
We are grateful to the Faculty of Informatics at the Università della Svizzera italiana (Switzerland) for having provided us with the valuable resources for the development and the testing of the AMiCI methodology. We especially thank Dr. Edgar Eduardo Rosales Rosero for the support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Adamoli, A. (2023). An Agile Concept Inventory Methodology to Detect Large Sets of Student Misconceptions in Programming Language Courses. In: Viberg, O., Jivet, I., Muñoz-Merino, P., Perifanou, M., Papathoma, T. (eds) Responsive and Sustainable Educational Futures. EC-TEL 2023. Lecture Notes in Computer Science, vol 14200. Springer, Cham. https://doi.org/10.1007/978-3-031-42682-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-42682-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42681-0
Online ISBN: 978-3-031-42682-7
eBook Packages: Computer ScienceComputer Science (R0)