How I Learned to Stop Worrying and Love ChatGPT
Pages 162 - 166
Abstract
In the dynamic landscape of software engineering, the emergence of ChatGPT-generated code signifies a distinctive and evolving paradigm in development practices. We delve into the impact of interactions with ChatGPT on the software development process, specifically analysing its influence on source code changes. Our emphasis lies in aligning code with ChatGPT conversations, separately analysing the user-provided context of the code and the extent to which the resulting code has been influenced by ChatGPT. Additionally, employing survival analysis techniques, we examine the longevity of ChatGPT-generated code segments in comparison to lines written traditionally. The goal is to provide valuable insights into the transformative role of ChatGPT in software development, illuminating its implications for code evolution and sustainability within the ecosystem.
References
[1]
Erik Bernhardsson. 2016. The half-life of code & the ship of Theseus. https://erikbern.com/2016/12/05/the-half-life-of-code.html Accessed: December 06, 2023.
[2]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877--1901. arXiv:2005.14165 [cs.CL]
[3]
Nathalia Moraes do Nascimento, Paulo S. C. Alencar, and Donald D. Cowan. 2023. Comparing Software Developers with ChatGPT: An Empirical Investigation. CoRR abs/2305.11837 (2023), 12 pages. arXiv:2305.11837 [cs.SE]
[4]
GitHub, Inc. and OpenAI. 2022. GitHub Copilot Documentation. https://docs.github.com/en/code-security/copilot Accessed: December 06, 2023.
[5]
Saki Imai. 2022. Is GitHub Copilot a Substitute for Human Pair-programming? An Empirical Study. In 44th IEEE/ACM International Conference on Software Engineering: Companion Proceedings, ICSE Companion 2022, Pittsburgh, PA, USA, May 22-24, 2022. ACM/IEEE, 319--321.
[6]
Mateusz Jaworski and Dariusz Piotrkowski. 2023. Study of software developers' experience using the Github Copilot Tool in the software development process. arXiv:2301.04991 [cs.SE]
[7]
E. L. Kaplan and Paul Meier. 1958. Nonparametric Estimation from Incomplete Observations. J. Amer. Statist. Assoc. 53, 282 (1958), 457--481.
[8]
Xian Liu. 2012. Survival analysis: models and applications. John Wiley & Sons.
[9]
Nhan Nguyen and Sarah Nadi. 2022. An Empirical Evaluation of GitHub Copilot's Code Suggestions. In 19th IEEE/ACM International Conference on Mining Software Repositories, MSR 2022, Pittsburgh, PA, USA, May 23-24, 2022. ACM, 1--5.
[10]
Piotr Przymus, Mikołaj Fejzer, Jakub Narębski, and Krzysztof Stencel. 2024. How I Learned to Stop Worrying and Love the ChatGPT - replication package. https://doi.org/10.6084/m9.figshare.24771117
[11]
John W Ratcliff, David Metzener, et al. 1988. Pattern matching: The gestalt approach. Dr. Dobb's Journal 13, 7 (1988), 46.
[12]
Michail Tsikerdekis. 2018. Persistent Code Contribution: A Ranking Algorithm for Code Contribution in Crowdsourced Software. Empirical Software Engineering 23, 4 (Aug. 2018), 1871--1894.
[13]
Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In CHI '22: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022 - 5 May 2022, Extended Abstracts, Simone D. J. Barbosa, Cliff Lampe, Caroline Appert, and David A. Shamma (Eds.). ACM, 332:1--332:7.
[14]
Guido Van Rossum and Fred L. Drake. 2009. Python 3 Reference Manual. CreateSpace, Scotts Valley, CA.
[15]
Guido Van Rossum and Fred L. Drake. 2023. Python 3 difflib. https://docs.python.org/3/library/difflib.html Accessed: December 06, 2023.
[16]
Tao Xiao, Christoph Treude, Hideaki Hata, and Kenichi Matsumoto. 2024. DevGPT: Studying Developer-ChatGPT Conversations. In Proceedings of the International Conference on Mining Software Repositories (MSR 2024).
[17]
Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz. 2023. Augmented intelligence in programming learning: Examining student views on the use of ChatGPT for programming learning. Computers in Human Behavior: Artificial Humans 1, 2 (2023), 100005.
Index Terms
- How I Learned to Stop Worrying and Love ChatGPT
Recommendations
Write me this Code: An Analysis of ChatGPT Quality for Producing Source Code
MSR '24: Proceedings of the 21st International Conference on Mining Software RepositoriesDevelopers nowadays are increasingly turning to large language models (LLMs) like ChatGPT to assist them with coding tasks, inspired by the promise of efficiency and the advanced capabilities they offer. However, this raises important questions about the ...
Using Code from ChatGPT: Finding Patterns in the Developers’ Interaction with ChatGPT
Reuse and Software QualityAbstractChatGPT can advise developers and provide code on how to fix bugs, add new features, refactor, reuse, and secure their code but currently, there is little knowledge about whether the developers trust ChatGPT’s responses and actually use the ...
Assessing the Readability of ChatGPT Code Snippet Recommendations: A Comparative Study
SBES '23: Proceedings of the XXXVII Brazilian Symposium on Software EngineeringDevelopers often rely on code search engines to find high-quality and reusable code snippets online, such as those available on Stack Overflow. Recently, ChatGPT, a language model trained for dialog tasks, has been gaining attention as a promising ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
788 pages
ISBN:9798400705878
DOI:10.1145/3643991
- Chair:
- Diomidis Spinellis,
- Program Chair:
- Alberto Bacchelli,
- Program Co-chair:
- Eleni Constantinou
Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MSR '24
Sponsor:
MSR '24: 21st International Conference on Mining Software Repositories
April 15 - 16, 2024
Lisbon, Portugal
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 37Total Downloads
- Downloads (Last 12 months)37
- Downloads (Last 6 weeks)3
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in