More Web Proxy on the site http://driver.im/

research-article

Open access

Plagiarism in the Age of Generative AI: Cheating Method Change and Learning Loss in an Intro to CS Course

Authors:

Colleen M. Lewis,

Craig ZillesAuthors Info & Claims

L@S '24: Proceedings of the Eleventh ACM Conference on Learning @ Scale

Pages 75 - 85

https://doi.org/10.1145/3657604.3662046

Published: 15 July 2024 Publication History

Abstract

Background: ChatGPT became widespread in early 2023 and enabled the broader public to use powerful generative AI, creating a new means for students to complete course assessments.

Purpose: In this paper, we explored the degree to which generative AI impacted the frequency and nature of cheating in a large introductory programming course. We also estimate the learning impact of students choosing to submit plagiarized work rather than their own work.

Methods: We identified a collection of markers that we believe are indicative of plagiarism in this course. We compare the estimated prevalence of cheating in the semesters before and during which ChatGPT became widely available. We use linear regression to estimate the impact of students' patterns of cheating on their final exam performance.

Findings: The patterns associated with these plagiarism markers suggest that the quantity of plagiarism increased with the advent of generative AI, and we see evidence of a shift from online plagiarism hubs (e.g., Chegg, CourseHero) to ChatGPT. In addition, we observe statistically significant learning losses proportional to the amount of presumed plagiarism, but there is no statistical difference on the proportionality between semesters.

Implications: Our findings suggest that unproctored exams become increasingly insecure and care needs to be taken to ensure the validity of summative assessments. More importantly, our results suggest that generative AI can be detrimental to students' learning. It seems necessary for educators to reduce the benefit of students using generative AI for counterproductive purposes.

References

[1]

Alex Aiken. 1994. MOSS: A System for Detecting Software Similarity. https://theory.stanford.edu/~aiken/moss/

[2]

Ibrahim Albluwi. 2019. Plagiarism in programming assessments: a systematic review. ACM Transactions on Computing Education (TOCE), Vol. 20, 1 (2019), 1--28.

[3]

Frank~B Baker. 2001. The basics of item response theory. ERIC.

[4]

Christina~Areizaga Barbieri, Dana Miller-Cotto, Sarah~N Clerjuste, and Kamal Chawla. 2023. A meta-analysis of the worked examples effect on mathematics performance. Educational Psychology Review, Vol. 35, 1 (2023), 11.

[5]

John Bransford, National Research~Council (U.S.), and National Research~Council (U.S.) (Eds.). 2000. How people learn: brain, mind, experience, and school expanded ed ed.). National Academy Press, Washington, D.C.

[6]

Peter~C Brown, Henry~L Roediger~III, and Mark~A McDaniel. 2014. Make it stick: The science of successful learning. Harvard University Press.

[7]

Debra~D Burke and Kenneth~J Sanney. 2018. Applying the fraud triangle to higher education: Ethical implications. J. Legal Stud. Educ., Vol. 35 (2018), 5.

[8]

Andrew~C Butler and Henry~L Roediger. 2008. Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & cognition, Vol. 36, 3 (2008), 604--616.

[9]

Binglin Chen, Sushmita Azad, Max Fowler, Matthew West, and Craig Zilles. 2020. Learning to cheat: quantifying changes in score advantage of unproctored assessments over time. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 197--206.

Digital Library

[10]

Binglin Chen, Matthew West, and Craig Zilles. 2018. How much randomization is needed to deter collaborative cheating on asynchronous exams?. In Proceedings of the fifth annual ACM conference on learning at scale. 1--10.

Digital Library

[11]

Freddie Choo and Kim Tan. 2008. The effect of fraud triangle factors on students' cheating behaviors. In Advances in accounting education. Vol., Vol. 9. Emerald Group Publishing Limited, 205--220.

[12]

Janice Connolly, Paula Lentz, Joline Morrison, et al. 2006. Using the business fraud triangle to predict academic dishonesty among business students. Academy of Educational Leadership Journal, Vol. 10, 1 (2006), 37.

[13]

Donald~R Cressey. 1953. Other people's money; a study of the social psychology of embezzlement. (1953).

[14]

Deborah~F Crown and M~Shane Spiller. 1998. Learning from the literature on collegiate cheating: A review of empirical research. Journal of business ethics, Vol. 17 (1998), 683--700.

[15]

Paul Denny, Viraj Kumar, and Nasser Giacaman. 2023. Conversing with copilot: Exploring prompt engineering for solving cs1 problems using natural language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. 1136--1142.

Digital Library

[16]

David~J Emerson and Kenneth~J Smith. 2022. Student use of homework assistance websites. Accounting Education, Vol. 31, 3 (2022), 273--293.

[17]

Joe Feldman. 2023. Grading for equity: What it is, why it matters, and how it can transform schools and classrooms. Corwin Press.

[18]

Kristin~Voelkl Finn and Michael~R Frone. 2004. Academic performance and cheating: Moderating role of school identification and self-efficacy. The journal of educational research, Vol. 97, 3 (2004), 115--121.

[19]

Valerie~J Haines, George~M Diekhoff, Emily~E LaBeff, and Robert~E Clark. 1986. College cheating: Immaturity, lack of commitment, and the neutralizing attitude. Research in Higher education, Vol. 25 (1986), 342--354.

[20]

Cindy~E Hmelo-Silver. 2004. Problem-based learning: What and how do students learn? Educational psychology review, Vol. 16 (2004), 235--266.

[21]

Thomas Lancaster and Codrin Cotarlan. 2021. Contract cheating by STEM students through a file sharing website: a Covid-19 pandemic perspective. International Journal for Educational Integrity, Vol. 17 (2021), 1--16.

[22]

Mark~M Lanier. 2006. Academic integrity and distance learning. Journal of criminal justice education, Vol. 17, 2 (2006), 244--261.

[23]

Victor~R. Lee, Rosalia~C. Zarate, Denise Pope, and Sarah~B. Miles. submitted, under review. Cheating in the Age of Generative AI: A High School Survey Study of Cheating Behaviors before and after the Release of ChatGPT. ( submitted, under review).

[24]

Donald~L McCabe and Linda~Klebe Trevino. 1997. Individual and contextual influences on academic dishonesty: A multicampus investigation. Research in higher education, Vol. 38 (1997), 379--396.

[25]

Bill McColl. 2023. Chegg Shares Plunge After Company Warns That ChatGPT Is Impacting Growth. https://www.investopedia.com/chegg-shares-plunge-after-company-warns-that-chatgpt-is-impacting-growth-7487968

[26]

Janet Metcalfe. 2017. Learning from errors. Annual review of psychology, Vol. 68 (2017), 465--489.

[27]

Paula~J Miles, Martin Campbell, and Graeme~D Ruxton. 2022. Why students cheat and how understanding this can help reduce the frequency of academic misconduct in higher education: a literature review. Journal of Undergraduate Neuroscience Education, Vol. 20, 2 (2022), A150.

[28]

Jaap~MJ Murre and Joeri Dros. 2015. Replication and analysis of Ebbinghaus' forgetting curve. PloS one, Vol. 10, 7 (2015), e0120644.

[29]

David~J Palazzo, Young-Jin Lee, Rasil Warnakulasooriya, and David~E Pritchard. 2010. Patterns, correlates, and reduction of homework copying. Physical Review Special Topics-Physics Education Research, Vol. 6, 1 (2010), 010104.

[30]

Jonathan Pierce and Craig Zilles. 2017. Investigating student plagiarism patterns and correlations to grades. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. 471--476.

Digital Library

[31]

Leo Porter. 2024. Learn AI-Assisted Python Programming: With Github Copilot and ChatGPT. Simon and Schuster.

[32]

Katherine~A. Rawson, John Dunlosky, and Sharon~M. Sciartelli. 2013. The Power of Successive Relearning: Improving Performance on Course Exams and Long-Term Retention. Educational Psychology Review, Vol. 25, 4 (Dec. 2013), 523--548. https://doi.org/10.1007/s10648-013--9240--4

[33]

Clara Sabbagh. 2021. Self-reported academic performance and academic cheating: Exploring the role of the perceived classroom (in) justice mediators. British Journal of Educational Psychology, Vol. 91, 4 (2021), 1517--1536.

[34]

Jaromir Savelka, Arav Agarwal, Marshall An, Chris Bogart, and Majd Sakr. 2023. Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses. In Proceedings of the 2023 ACM Conference on International Computing Education Research.

Digital Library

[35]

Daniel~L. Schwartz, Jessica~M. Tsang, and Kristen~P. Blair. 2016. The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them first edition ed.). W.W. Norton & Company, Inc., New York, NY. OCLC: 954134221.

[36]

Valerie~J Shute. 2008. Focus on formative feedback. Review of educational research, Vol. 78, 1 (2008), 153--189.

[37]

Ben Skudder and Andrew Luxton-Reilly. 2014. Worked examples in computer science. In Proceedings of the Sixteenth Australasian Computing Education Conference-Volume 148. 59--64.

Digital Library

[38]

Robert Stickgold and Matthew~P Walker. 2005. Memory consolidation and reconsolidation: what is the role of sleep? Trends in neurosciences, Vol. 28, 8 (2005), 408--415.

[39]

Fabienne~M Van~der Kleij, Remco~CW Feskens, and Theo~JHM Eggen. 2015. Effects of feedback in a computer-based learning environment on students' learning outcomes: A meta-analysis. Review of educational research, Vol. 85, 4 (2015), 475--511.

[40]

Tamara Van~Gog, Liesbeth Kester, and Fred Paas. 2011. Effects of worked examples, example-problem, and problem-example pairs on novices' learning. Contemporary Educational Psychology, Vol. 36, 3 (2011), 212--218.

[41]

Matthew West, Geoffrey~L Herman, and Craig Zilles. 2015. Prairielearn: Mastery-based online problem solving with adaptive scoring and recommendations driven by machine learning. In 2015 ASEE Annual Conference & Exposition. 26--1238.

[42]

Hongwei Yu, Perry~L Glanzer, Byron~R Johnson, Rishi Sriram, and Brandon Moore. 2018. Why college students cheat: A conceptual model of five factors. The Review of Higher Education, Vol. 41, 4 (2018), 549--576.

[43]

Craig Zilles, Matthew West, David Mussulman, and Tim Bretl. 2018. Making testing less trying: Lessons learned from operating a Computer-Based Testing Facility. In 2018 IEEE Frontiers in Education Conference (FIE). IEEE, 1--9.

Digital Library

[44]

Craig~B Zilles, Matthew West, Geoffrey~L Herman, and Timothy Bretl. 2019. Every University Should Have a Computer-Based Testing Facility. In CSEDU (1). 414--420.

Cited By

Zilles CZhao CChen YMatthews EWest MDorodchi MZhange MCooper S(2024)A Case for Bayesian GradingProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 110.1145/3649165.3703624(275-278)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649165.3703624
Akçapınar GSidan E(2024)AI chatbots in programming education: guiding success or encouraging plagiarismDiscover Artificial Intelligence10.1007/s44163-024-00203-74:1Online publication date: 21-Nov-2024
https://doi.org/10.1007/s44163-024-00203-7

Index Terms

Plagiarism in the Age of Generative AI: Cheating Method Change and Learning Loss in an Intro to CS Course
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. K-12 education

Recommendations

How should we change teaching and assessment in response to increasingly powerful generative Artificial Intelligence? Outcomes of the ChatGPT teacher survey
Abstract
There has been widespread media commentary about the potential impact of generative Artificial Intelligence (AI) such as ChatGPT on the Education field, but little examination at scale of how educators believe teaching and assessment should change ...
Cheating and plagiarism: perceptions and practices of first year IT students

A study of cheating and plagiarism using a scenario based format, has provided an insight into attitudes towards questionable work practices of first year information technology students' within the School of Computer Science and Software Engineering of ...
Cheating and plagiarism: perceptions and practices of first year IT students
ITiCSE '02: Proceedings of the 7th annual conference on Innovation and technology in computer science education

A study of cheating and plagiarism using a scenario based format, has provided an insight into attitudes towards questionable work practices of first year information technology students' within the School of Computer Science and Software Engineering of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

L@S '24: Proceedings of the Eleventh ACM Conference on Learning @ Scale

July 2024

582 pages

ISBN:9798400706332

DOI:10.1145/3657604

General Chair:
David Joyner
Georgia Tech, USA
,
Program Chairs:
Min Kyu Kim
Georgia State University, USA
,
Xu Wang
University of Michigan, USA
,
Meng Xia
Texas A&M University, USA

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

L@S '24

L@S '24: Eleventh ACM Conference on Learning @ Scale

July 18 - 20, 2024

GA, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 117 of 440 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
463
Total Downloads

Downloads (Last 12 months)463
Downloads (Last 6 weeks)193

Reflects downloads up to 10 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zilles CZhao CChen YMatthews EWest MDorodchi MZhange MCooper S(2024)A Case for Bayesian GradingProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 110.1145/3649165.3703624(275-278)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649165.3703624
Akçapınar GSidan E(2024)AI chatbots in programming education: guiding success or encouraging plagiarismDiscover Artificial Intelligence10.1007/s44163-024-00203-74:1Online publication date: 21-Nov-2024
https://doi.org/10.1007/s44163-024-00203-7

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents