Novice Learners of Programming and Generative AI - Prior Knowledge Matters
Article No.: 51, Pages 1 - 2
Abstract
With the broad availability of Generative AI (GenAI), introductory programming education is starting to change. At Nuremberg Tech, we observed the doubling of failure rates to approximately 50% in the first semester course “Procedural Programming” across students of all study programs. Due to these exam results in winter 2023/24, we conducted a pilot study to gather students’ use of GenAI tools, their exam results, and prior programming education and experience. The results imply significant differences of students’ use of GenAI tools depending on their prior programming education. We will therefore extend the investigation in winter term 2024/25.
References
[1]
Sara Amani, Lance White, Trini Balart, Laksha Arora, Dr. Kristi J. Shryock, Dr. Kelly Brumbelow, and Dr. Karan L. Watson. 2023. Generative AI Perceptions: A Survey to Measure the Perceptions of Faculty, Staff, and Students on Generative AI Tools in Academia. arxiv:https://arXiv.org/abs/2304.14415 [cs.HC]
[2]
Imen Azaiz, Natalie Kiesler, and Sven Strickroth. 2024. Feedback-Generation for Programming Exercises With GPT-4. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (Milan, Italy) (ITiCSE 2024). ACM, New York, 31–37.
[3]
Brett A Becker, Michelle Craig, Paul Denny, Hieke Keuning, Natalie Kiesler, Juho Leinonen, Andrew Luxton-Reilly, James Prather, and Keith Quille. 2023. Generative AI in Introductory Programming. (2023). https://csed.acm.org/wp-content/uploads/2023/12/Generative-AI-Nov-2023-Version.pdf
[4]
Norbert Forman, József Udvaros, and Mihály Szilárd Avornicului. 2023. ChatGPT: A new study tool shaping the future for high school students. International Journal of Advanced Natural Sciences and Engineering Researches 7, 4 (2023), 95–102.
[5]
Virginia Grande, Natalie Kiesler, and Maria Andreina Francisco Rodriguez. 2024. Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics. In Proceedings of the 2024 Conference on Innovation and Technology in Computer Science Education V. 2 (Milan, Italy) (ITiCSE 2024). ACM, New York.
[6]
Dhanya Jayagopal, Justin Lubin, and Sarah E Chasins. 2022. Exploring the learnability of program synthesizers by novice programmers. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–15.
[7]
Natalie Kiesler, Dominic Lohr, and Hieke Keuning. 2024. Exploring the Potential of Large Language Models to Generate Formative Programming Feedback. In 2023 IEEE Frontiers in Education Conference (FIE). 1–5.
[8]
Juho Leinonen, Paul Denny, Stephen MacNeil, Sami Sarsa, Seth Bernstein, Joanne Kim, Andrew Tran, and Arto Hellas. 2023. Comparing Code Explanations Created by Students and Large Language Models. In Proc. ITiCSE. 124–130.
[9]
Juho Leinonen, Arto Hellas, Sami Sarsa, Brent Reeves, Paul Denny, James Prather, and Brett A. Becker. 2023. Using Large Language Models to Enhance Programming Error Messages. In Proc. SIGCSE. ACM.
[10]
Stephen MacNeil, Andrew Tran, Arto Hellas, Joanne Kim, Sami Sarsa, Paul Denny, Seth Bernstein, and Juho Leinonen. 2023. Experiences from Using Code Explanations Generated by Large Language Models in a Web Software Development E-Book. In Proc. SIGCSE TS. 931–937.
[11]
James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton-Reilly, Stephen MacNeil, Andrew Petersen, Raymond Pettit, Brent N. Reeves, and Jaromir Savelka. 2023. The Robots Are Here: Navigating the Generative AI Revolution in Computing Education. In Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education (Turku, Finland). ACM, New York, 108–159.
[12]
James Prather, Juho Leinonen, Natalie Kiesler, Jamie Gorson Benario, Sam Lau, Stephen MacNeil, Narges Norouzi, Simone Opel, Virginia Pettit, Leo Porter, Brent N. Reeves, Jaromir Savelka, David H. Smith, Sven Strickroth, and Daniel Zingaro. 2024. How Instructors Incorporate Generative AI into Teaching Computing. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 2 (Milan, Italy) (ITiCSE 2024). ACM, New York, 771–772.
[13]
Raghu Raman, Santanu Mandal, Payel Das, Tavleen Kaur, JP Sanjanasri, and Prema Nedungadi. 2023. University students as early adopters of ChatGPT: Innovation Diffusion Study. (2023).
[14]
Simon Roderus and Uwe Wienkop. 2014. Verbesserung der Bestehensquoten durch ein Peer Assessment-Pflichtpraktikum. HDI 2014–Gestalten von Übergängen (2014), 45.
[15]
Andreas Scholl and Natalie Kiesler. 2024. How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course.
[16]
Andreas Scholl, Daniel Schiffner, and Natalie Kiesler. 2024. Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT. In Proceedings of DELFI 2024, Sandra Schulz and Natalie Kiesler (Eds.). Gesellschaft für Informatik e.V., Bonn, 63–79.
[17]
Alexander Tillmann, Detlef Krömker, Florian Horn, and Torsten Gattinger. 2018. Analysing & Predicting Students Performance in an Introductory Computer Science Course. In HDI Proceedings. 29–46.
[18]
Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. ACM, New York, 1–7.
Index Terms
- Novice Learners of Programming and Generative AI - Prior Knowledge Matters
Recommendations
Ne-course for learning programming
TEEM '16: Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing MulticulturalityDifficulties in learning programming are a constant concern in engineering courses. In many research studies involving the learning programming must of the solutions presented, from the beginning of the first programming languages, was to apply different ...
A Four-year Study of a Placement Exam for an Introductory Programming Course
SIGCSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1Introductory programming courses can be challenging due to heterogeneous student backgrounds. While novice programmers find even basic contents very demanding, experienced programmers are not sufficiently challenged. Having both novice and experienced ...
Teaching programming to liberal arts students: a narrative media approach
Proceedings of the 8th annual conference on Innovation and technology in computer science educationIn this paper we present a new learning environment to be used in an introductory programming course for students that are non-majors in computer science, more precisely for multimedia students with a liberal arts background.Media-oriented programming ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 Copyright held by the owner/author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 November 2024
Check for updates
Author Tags
Qualifiers
- Poster
Conference
Koli Calling '24
Koli Calling '24: 24th Koli Calling International Conference on Computing Education Research
November 12 - 17, 2024
Koli, Finland
Acceptance Rates
Overall Acceptance Rate 80 of 182 submissions, 44%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 46Total Downloads
- Downloads (Last 12 months)46
- Downloads (Last 6 weeks)46
Reflects downloads up to 21 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format