More Web Proxy on the site http://driver.im/

research-article

Correlating Students' Class Performance Based on GitHub Metrics: A Statistical Study

Authors:

Edward GehringerAuthors Info & Claims

ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1

Pages 526 - 532

https://doi.org/10.1145/3587102.3588799

Published: 30 June 2023 Publication History

Abstract

What skills does a student need to succeed in a programming class? Ostensibly, previous programming experience may affect a student's performance. Most past studies on this topic use self-reporting questionnaires to query students about their programming experience. This paper presents a novel, unified, and replicable way to measure previous programming experience using students' pre-class GitHub contributions. To our knowledge, we are the first to use GitHub contributions in this way. We conducted a comprehensive statistical study of students in an object-oriented design and development class from 2017 to 2022 (n = 751) to explore the relationships between GitHub contributions (commits, comments, pull requests, etc.) and students' performance on exams, projects, designs, etc. in the class. Several kinds of contributions were shown to have statistically significant correlations with performance in the class. A set of two-samplet -tests demonstrate statistical significance of the difference between the means of some contributions from the high-performing and low-performing groups.

References

[1]

Haldun Akoglu. 2018. User's guide to correlation coefficients. Turkish Journal of Emergency Medicine 18 (2018), 91 -- 93.

[2]

Susan Bergin and Ronan Reilly. 2005. Programming: Factors That Influence Success. In Proceedings of the 36th SIGCSE Technical Symposium on Computer Science Education (St. Louis, Missouri, USA) (SIGCSE '05). Association for Computing Machinery, New York, NY, USA, 411--415. https://doi.org/10.1145/1047344.1047480

Digital Library

[3]

Nicholas A. Bowman, Lindsay Jarratt, K.C. Culver, and Alberto Maria Segre. 2019. How Prior Programming Experience Affects Students' Pair Programming Experiences and Outcomes. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE '19). Association for Computing Machinery, New York, NY, USA, 170--175. https://doi.org/10.1145/3304221.3319781

Digital Library

[4]

Pat Byrne and Gerry Lyons. 2001. The Effect of Student Attributes on Success in Programming. In Proceedings of the 6th Annual Conference on Innovation and Technology in Computer Science Education (Canterbury, United Kingdom) (ITiCSE '01). Association for Computing Machinery, New York, NY, USA, 49--52. https: //doi.org/10.1145/377435.377467

Digital Library

[5]

Jialin Cui, Ruochi Li, Kaida Lou, Chengyuan Liu, Yunkai Xiao, Qinjin Jia, Edward Gehringer, and Runqiu Zhang. 2022. Can Pre-Class Github Contributions Predict Success by Student Teams?. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Software Engineering Education and Training (Pittsburgh, Pennsylvania) (ICSE-SEET '22). Association for Computing Machinery, New York, NY, USA, 40--49. https://doi.org/10.1145/3510456.3514144

Digital Library

[6]

docs.github.com. 2022. GraphQL API. Retrieved August 10, 2022 from https: //docs.github.com/en/graphql

[7]

education.github.com. 2022. You train the next generation of software developers. Retrieved August 10, 2022 from https://education.github.com/teachers

[8]

Madeline Endres,WestleyWeimer, and Amir Kamil. 2021. An Analysis of Iterative and Recursive Problem Performance. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE '21). Association for Computing Machinery, New York, NY, USA, 321--327. https: //doi.org/10.1145/3408877.3432391

Digital Library

[9]

Lidia Feklistova, Marina Lepp, and Piret Luik. 2021. Learners' Performance in a MOOC on Programming. Education Sciences 11, 9 (2021), 521.

[10]

David Gray. 2017. Doing Research in the Real World (4th ed.). SAGE Publications Ltd, London.

[11]

Dianne Hagan and Selby Markham. 2000. Does It Help to Have Some Programming Experience before Beginning a Computing Degree Program? SIGCSE Bull. 32, 3 (jul 2000), 25--28. https://doi.org/10.1145/353519.343063

Digital Library

[12]

Zoe A Kersteen, Marcia C Linn, Michael Clancy, and Curtis Hardyck. 1988. Previous experience and the learning of computer programming: The computer helps those who help themselves. Journal of Educational Computing Research 4, 3 (1988), 321--333.

[13]

Paul Krill. 2020. GitHub expects 100 million software developers by 2025. Retrieved August 10, 2022 from https://www.infoworld.com/article/3599874/githubexpects- 100-million-software-developers-by-2025.html

[14]

madnight.github.io. 2022. popular languages on GitHub. Retrieved August 10, 2022 from https://madnight.github.io/githut/#/pull_requests/2022/1

[15]

Wolfram Research. accessed on Janu 23, 2023. Bonferroni Correction. https: //mathworld.wolfram.com/BonferroniCorrection.html.

[16]

William R Rice. 1989. Analyzing tables of statistical tests. Evolution 43, 1 (1989), 223--225.

[17]

Jenny Rowley. 2014. Designing and using research questionnaires. Management Research Review 37, 3 (2014), 308--330. https://doi.org/10.1108/MRR-02--2013-0027

[18]

Christopher Watson, Frederick W.B. Li, and Jamie L. Godwin. 2014. No Tests Required: Comparing Traditional and Dynamic Predictors of Programming Success. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (Atlanta, Georgia, USA) (SIGCSE '14). Association for Computing Machinery, New York, NY, USA, 469--474. https://doi.org/10.1145/2538862.2538930

Digital Library

[19]

Susan Wiedenbeck, Deborah LaBelle, and Vennila N. R. Kain. 2004. Factors affecting course outcomes in introductory programming. In Proceedings of the 16th Annual Workshop of the Psychology of Programming Interest Group, PPIG 2004, Carlow, Ireland, April 5--7, 2004. Psychology of Programming Interest Group, Carlow, Ireland, 11. https://ppig.org/papers/2004-ppig-16th-wiedenbeck/

[20]

Chris Wilcox and Albert Lionelle. 2018. Quantifying the Benefits of Prior Programming Experience in an Introductory Computer Science Course. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE '18). Association for Computing Machinery, New York, NY, USA, 80--85. https://doi.org/10.1145/3159450.3159480

Digital Library

Cited By

Cui JZhou FLiu CJia QSong YGehringer EMonga MLonati VBarendsen ESheard JPaterson J(2024)Utilizing the Constrained K-Means Algorithm and Pre-Class GitHub Contribution Statistics for Forming Student TeamsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653634(569-575)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653634
Cui JZhang RLi RZhou FSong YGehringer EMonga MLonati VBarendsen ESheard JPaterson J(2024)A Comparative Analysis of GitHub Contributions Before and After An OSS Based Software Engineering ClassProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653535(576-582)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653535
Cui JZhang RLi RZhou FSong YGehringer EStephenson BStone JBattestilli LRebelsky SShoop L(2024)How Pre-class Programming Experience Influences Students' Contribution to Their Team Project: A Statistical StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630870(255-261)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630870
Show More Cited By

Index Terms

Correlating Students' Class Performance Based on GitHub Metrics: A Statistical Study
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Software engineering education
2. Software and its engineering
  1. Software creation and management
    1. Collaboration in software development
      1. Open source model
      2. Programming teams
    2. Software development techniques
      1. Object oriented development

Recommendations

Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects
Assessing team software development projects is notoriously difficult and typically based on subjective metrics. To help make assessments more rigorous, we conducted an empirical study to explore relationships between subjective metrics based on peer and ...
A Comparative Analysis of GitHub Contributions Before and After An OSS Based Software Engineering Class
ITiCSE 2024: Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1

This study presents a comparative analysis of contributions to GitHub by students before and after participating in a Software Engineering class based on Open Source Software (OSS). The primary objective is to understand the influence of formal software ...
How Pre-class Programming Experience Influences Students' Contribution to Their Team Project: A Statistical Study
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1

Group or team projects are an essential component of the software engineering curriculum. Earlier studies have explored how prior programming experience influences students' team project performance and overall class performance in software engineering. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1

June 2023

694 pages

ISBN:9798400701382

DOI:10.1145/3587102

General Chairs:
Mikko-Jussi Laakso
University of Turku, Finland
,
Mattia Monga
University of Milan, Italy
,
Program Chairs:
Simon
Unaffiliated, Australia
,
Judithe Sheard
Monash University, Australia

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE 2023

Sponsor:

SIGCSE

ITiCSE 2023: Innovation and Technology in Computer Science Education

July 7 - 12, 2023

Turku, Finland

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
77
Total Downloads

Downloads (Last 12 months)35
Downloads (Last 6 weeks)3

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cui JZhou FLiu CJia QSong YGehringer EMonga MLonati VBarendsen ESheard JPaterson J(2024)Utilizing the Constrained K-Means Algorithm and Pre-Class GitHub Contribution Statistics for Forming Student TeamsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653634(569-575)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653634
Cui JZhang RLi RZhou FSong YGehringer EMonga MLonati VBarendsen ESheard JPaterson J(2024)A Comparative Analysis of GitHub Contributions Before and After An OSS Based Software Engineering ClassProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653535(576-582)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653535
Cui JZhang RLi RZhou FSong YGehringer EStephenson BStone JBattestilli LRebelsky SShoop L(2024)How Pre-class Programming Experience Influences Students' Contribution to Their Team Project: A Statistical StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630870(255-261)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630870
Cui JZhang RZhou FLi RSong YGehringer E(2024)How Much Effort Do You Need to Expend on a Technical Interview? A Study of LeetCode Problem Solving Statistics2024 36th International Conference on Software Engineering Education and Training (CSEE&T)10.1109/CSEET62301.2024.10663022(1-10)Online publication date: 29-Jul-2024
https://doi.org/10.1109/CSEET62301.2024.10663022
Cui JZhou FZhang RLi RLiu CGehringer E(2023)Predicting Students' Software Engineering Class Performance with Machine Learning and Pre-Class GitHub Metrics2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343357(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343357

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents