More Web Proxy on the site http://driver.im/

research-article

Assessing Individual Contributions to Software Engineering Projects with Git Logs and User Stories

Author:

Kevin BuffardiAuthors Info & Claims

SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

Pages 650 - 656

https://doi.org/10.1145/3328778.3366948

Published: 26 February 2020 Publication History

Abstract

Software Engineering courses often incorporate large-scale projects with collaboration between students working in teams. However, it is difficult to objectively assess individual students when their projects are a product of collaborative efforts. This study explores measurements of individuals' contributions to their respective teams.

I analyzed ten Software Engineering team projects (n=42) and evaluations of individual contributions using automated evaluation of the version control system history (Git logs) and user stories completed on their project management (Kanban) boards. Unique insights from meta-data within the Git history and Kanban board user stories reveal complicated relationships between these measurements and traditional assessments, such as peer review and subjective instructor evaluation. From the results, I suggest supplementing and validating traditional assessments with insights from individuals' commit history and user story contributions.

References

[1]

Howard Abikoff, Mary Courtney, William E. Pelham, and Harold S. Koplewicz. 1993. Teachers' ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology 21, 5 (01 Oct 1993), 519--533. https://doi.org/10.1007/BF00916317

[2]

ACM. 2013. Computer Science 2013: Curriculum Guidelines for Undergraduate Programs in Computer Science. http://www.acm.org/education/curricula-recommendations

[3]

Francesca Arcelli Fontana and Claudia Raibulet. 2017. Students' Feedback in Using GitHub in a Project Development for a Software Engineering Course. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 380--380. https://doi.org/10.1145/3059009.3072984

Digital Library

[4]

Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas. 2001. Agile Manifesto. http://agilemanifesto.org/

[5]

Grant Braught, John Maccormick, James Bowring, Quinn Burke, Barbara Cutler, David Goldschmidt, Mukkai Krishnamoorthy, Wesley Turner, Steven Huss- Lederman, Bonnie Mackellar, and Allen Tucker. 2018. A Multi-Institutional Perspective on H/FOSS Projects in the Computing Curriculum. ACM Trans. Comput. Educ. 18, 2, Article 7 (July 2018), 31 pages. https://doi.org/10.1145/3145476

Digital Library

[6]

Bernd Bruegge, Stephan Krusche, and Lukas Alperowitz. 2015. Software Engineering Project Courses with Industrial Clients. Trans. Comput. Educ. 15, 4, Article 17 (Dec. 2015), 31 pages. https://doi.org/10.1145/2732155

[7]

Kevin Buffardi. 2018. Tech Startup Learning Activities: A Formative Evaluation. In IEEE/ACM International Workshop on Software Engineering Education for Millennials (SEEM). 24--31.

Digital Library

[8]

Kevin Buffardi. 2019. gitlog2csv. https://github.com/kbuffardi/gitlog2csv

[9]

Kevin Buffardi, Colleen Robb, and David Rahn. 2017. Learning Agile with Tech Startup Software Engineering Projects. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 28--33. https://doi.org/10.1145/3059009.3059063

Digital Library

[10]

Kevin Buffardi, Colleen Robb, and David Rahn. 2017. Tech Startups: Realistic Software Engineering Projects with Interdisciplinary Collaboration. J. Comput. Sci. Coll. 32, 4 (April 2017), 93--98. http://dl.acm.org/citation.cfm?id=3055338.3055355

Digital Library

[11]

Donald T. Campbell. 1979. Assessing the impact of planned social change. Evaluation and Program Planning 2, 1 (1979), 67 -- 90. https://doi.org/10.1016/0149--7189(79)90048-X

[12]

Nicole Clark. 2005. Evaluating Student Teams Developing Unique Industry Projects. In Proceedings of the 7th Australasian Conference on Computing Education - Volume 42 (ACE '05). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 21--30. http://dl.acm.org/citation.cfm?id=1082424.1082428

Digital Library

[13]

N. M. Devadiga. 2017. Software Engineering Education: Converging with the Startup Industry. In 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE T). 192--196. https://doi.org/10.1109/CSEET.2017.38

[14]

Heidi J.C. Ellis and Gregory W. Hislop. 2016. Pathways to Student Learning Within HFOSS. In Proceedings of the 17th Annual Conference on Information Technology Education (SIGITE '16). ACM, New York, NY, USA, 168--168. https://doi.org/10.1145/2978192.2978242

[15]

F. Fagerholm and A. Vihavainen. 2013. Peer assessment in experiential learning Assessing tacit and explicit skills in agile software engineering capstone projects. In 2013 IEEE Frontiers in Education Conference (FIE). 1723--1729. https://doi.org/10.1109/FIE.2013.6685132

[16]

Vahid Garousi, Kai Petersen, and Baris Ozkan. 2016. Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review. Information and Software Technology 79 (2016), 106 -- 127. https://doi.org/10.1016/j.infsof.2016.07.006

Digital Library

[17]

GitHub. 2019. The world's leading software development platform - GitHub. https://github.com/ Accessed on August 2019.

[18]

Nicole Herbert. 2007. Quantitative Peer Assessment: Can Students Be Objective?. In Proceedings of the Ninth Australasian Conference on Computing Education - Volume 66 (ACE '07). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 63--71. http://dl.acm.org/citation.cfm?id=1273672.1273680

[19]

Bibb Latane, Kipling Williams, and Stephen Harkins. 1979. Many hands make light the work: The causes and consequences of social loafing. Journal of Personality and Social Psychology 37 (1979), 822--832. Issue 6. https://doi.org/10.1037/0022--3514.37.6.822

[20]

Richard A. Layton, Matthew W. Ohland, and Hal Pomeranz. 2007. Software For Student Team Formation And Peer Evaluation: Catme Incorporates Team Maker.

[21]

Robert C. Martin. 2006. Agile Software Development: Principles, Patterns, and Practices. Prentice Hall.

Digital Library

[22]

Nancy R. Mead. 2015. Industry/University Collaboration in Software Engineering Education: Refreshing and Retuning Our Strategies. In Proceedings of the 37th International Conference on Software Engineering - Volume 2 (ICSE '15). IEEE Press, Piscataway, NJ, USA, 273--275. http://dl.acm.org/citation.cfm?id=2819009.2819050

[23]

Christian Murphy, Kevin Buffardi, Josh Dehlinger, Lynn Lambert, and Nanette Veilleux. 2017. Community Engagement with Free and Open Source Software. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17). ACM, New York, NY, USA, 669--670. https://doi.org/10.1145/3017680.3017682

Digital Library

[24]

Natalja Nikitina, Mira Kajko-Mattsson, and Magnus Stråle. 2012. From Scrum to Scrumban: A Case Study of a Process Transition. In Proceedings of the International Conference on Software and System Process (ICSSP '12). IEEE Press, Piscataway, NJ, USA, 140--149. http://dl.acm.org/citation.cfm?id=2664360.2664382

[25]

Matthew Ohland, Misty Loughry, Rufus L. Carter, Lisa Bullard, Richard Felder, Cynthia Finelli, Richard Layton, and Douglas Schmucker. 2005. Developing a Peer Evaluation Instrument that is Simple, Reliable, and Valid. Proceedings 2005 American Society of Engineering Education Conference and Exposition Portland Oregon (01 2005).

[26]

MatthewW. Ohland, Misty L. Loughry, David J.Woehr, Lisa G. Bullard, Richard M. Felder, Cynthia J. Finelli, Richard A. Layton, Hal R. Pomeranz, and Douglas G. Schmucker. 2012. The Comprehensive Assessment of Team Member Effectiveness: Development of a Behaviorally Anchored Rating Scale for Self- and Peer Evaluation. Academy of Management Learning & Education 11, 4 (2012), 609--630. https://doi.org/10.5465/amle.2010.0177

[27]

Helen Parker and Mike Holcombe. 1999. Campus-based Industrial Software Projects: Risks and Rewards. SIGCSE Bull. 31, 3 (June 1999), 189--. https://doi.org/10.1145/384267.305935

[28]

Gustavo Pinto, Clarice Ferreira, Cleice Souza, Igor Steinmacher, and Paulo Meirelles. 2019. Training Software Engineers Using Open-source Software: The Students' Perspective. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET '19). IEEE Press, Piscataway, NJ, USA, 147--157. https://doi.org/10.1109/ICSE-SEET.2019.00024

Digital Library

[29]

Arnold Rosenbloom, Sadia Sharmin, and Andrew Wang. 2017. GIT: Pedagogy, Use and Administration in Undergraduate CS. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 82--83. https://doi.org/10.1145/3059009.3072980

Digital Library

[30]

Frank W. Schneider, Jamie A. Gruman, and Larry M. Coutts. 2012. Applied Social Psychology: Understanding and Addressing Social and Practical Problems. SAGE.

[31]

Therese Mary Smith, Robert McCartney, Swapna S. Gokhale, and Lisa C. Kaczmarczyk. 2014. Selecting Open Source Software Projects to Teach Software Engineering. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14). ACM, New York, NY, USA, 397--402. https://doi.org/10.1145/2538862.2538932

[32]

Jeff Sutherland. 2018. Impediments of Performance Appraisals. https://www.scruminc.com/performance-appraisals/ Accessed November 2019.

[33]

Anya Tafliovich, Andrew Petersen, and Jennifer Campbell. 2015. On the Evaluation of Student Team Software Development Projects. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). ACM, New York, NY, USA, 494--499. https://doi.org/10.1145/2676723.2677223

Digital Library

[34]

Linda van den Bergh, Eddie Denessen, Lisette Hornstra, Marinus Voeten, and Rob W. Holland. 2010. The Implicit Prejudiced Attitudes of Teachers: Relations to Teacher Expectations and the Ethnic Achievement Gap. American Educational Research Journal 47, 2 (2010), 497--527. https://doi.org/10.3102/0002831209353594 arXiv:https://doi.org/10.3102/0002831209353594

Cited By

de Miranda FFerrao RSoler DVieira Graglia MDorodchi MZhang MCooper S(2024)LLM-based Individual Contribution Summarization in Software ProjectsProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 210.1145/3649409.3691092(307-308)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649409.3691092
Schauer LStewart RMaarek MRoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Integrating Canvas and GitLab to Enrich Learning ProcessesProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640056(180-190)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640056
Guttmann MKaraka AHelic DStephenson BStone JBattestilli LRebelsky SShoop L(2024)Attribution of Work in Programming Teams with Git ReporterProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630785(436-442)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630785
Show More Cited By

Index Terms

Assessing Individual Contributions to Software Engineering Projects with Git Logs and User Stories
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
        Project management techniques
2. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods
        Agile software development
  2. Software notations and tools
    1. Software configuration management and version control systems

Recommendations

"Leagile" software development

In recent years there has been a noticeable shift in attention from those who use agile software development toward lean software development, often labelled as a shift "from agile to lean". However, the reality may not be as simple or linear as this ...
Learning Agile with Tech Startup Software Engineering Projects
ITiCSE '17: Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

The Tech Startup model is an approach to learning software engineering methods by partnering with students studying entrepreneurship to collaborate on real software products. Agile software development methods align with Lean Startup practices so that ...
Software Engineering Practices and Methods in the Game Development Industry
CHI PLAY '19 Extended Abstracts: Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts

Applying current software engineering practices in the game development industry is a rapidly growing but under researched area. Whether game development studios align to traditional software engineering practices such as agile methodologies to develop ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

February 2020

1502 pages

ISBN:9781450367936

DOI:10.1145/3328778

General Chairs:
Jian Zhang
Texas Woman's University, USA
,
Mark Sherriff
University of Virginia, USA
,
Program Chairs:
Sarah Heckman
North Carolina State University, USA
,
Pamela Cutter
Kalamazoo College, USA
,
Alvaro Monge
California State University, Long Beach, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 February 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '20

Sponsor:

SIGCSE

SIGCSE '20: The 51st ACM Technical Symposium on Computer Science Education

March 11 - 14, 2020

OR, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025

Sponsor:
sigcse

The 56th ACM Technical Symposium on Computer Science Education

February 26 - March 1, 2025

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
610
Total Downloads

Downloads (Last 12 months)99
Downloads (Last 6 weeks)5

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

de Miranda FFerrao RSoler DVieira Graglia MDorodchi MZhang MCooper S(2024)LLM-based Individual Contribution Summarization in Software ProjectsProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 210.1145/3649409.3691092(307-308)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649409.3691092
Schauer LStewart RMaarek MRoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Integrating Canvas and GitLab to Enrich Learning ProcessesProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640056(180-190)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640056
Guttmann MKaraka AHelic DStephenson BStone JBattestilli LRebelsky SShoop L(2024)Attribution of Work in Programming Teams with Git ReporterProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630785(436-442)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630785
Vossen PAjit S(2024)Measurement Models for Group-Peer Assessment of Project-Based Learning in Software Engineering2024 International Symposium on Computer Science and Educational Technology (ISCSET)10.1109/ISCSET58624.2024.10808114(1-5)Online publication date: 17-Jul-2024
https://doi.org/10.1109/ISCSET58624.2024.10808114
Patani PTiwari SRathore S(2024)The impact of GitHub on students' learning and engagement in a software engineering courseComputer Applications in Engineering Education10.1002/cae.2277532:5Online publication date: 18-Jun-2024
https://doi.org/10.1002/cae.22775
Peng TLiang YWu WRen JPengrui ZPu YWilliams BChen YNeville J(2023)CLGTProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i13.26893(15947-15954)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i13.26893
Chen WChen HHsueh Y(2023)Work in Progress: GitHub-Driven Learning: A Case Study in Agile-Infused Software Design CoursesProceedings of the 2023 8th International Conference on Information Systems Engineering10.1145/3641032.3641039(13-18)Online publication date: 16-Dec-2023
https://dl.acm.org/doi/10.1145/3641032.3641039
Malmi LSheard JSinclair JKinnunen PSimon (2023)Domain-Specific Theories of Teaching Computing: Do they Inform Practice?Proceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631810(1-15)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631810
Hundhausen CConrad PAdesope OTariq A(2023)Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development ProjectsACM Transactions on Computing Education10.1145/359359223:3(1-23)Online publication date: 22-Jul-2023
https://dl.acm.org/doi/10.1145/3593592
Hundhausen CConrad PAdesope OTariq ASbai SLu ADoyle MStephenson BDorn BSoh LBattestilli L(2023)Investigating Reflection in Undergraduate Software Development TeamsProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 110.1145/3545945.3569790(743-749)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3545945.3569790
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents