More Web Proxy on the site http://driver.im/

research-article

Pragmatic Software Testing Education

Authors:

Maurício Aniche,

Felienne Hermans,

Arie van DeursenAuthors Info & Claims

SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Pages 414 - 420

https://doi.org/10.1145/3287324.3287461

Published: 22 February 2019 Publication History

Abstract

Software testing is an important topic in software engineering education, and yet highly challenging from an educational perspective: students are required to learn several testing techniques, to be able to distinguish the right technique to apply, to evaluate the quality of their test suites, and to write maintainable test code. In this paper, we describe how we have been adding a pragmatic perspective to our software testing course, and explore students' common mistakes, hard topics to learn, favourite learning activities, and challenges they face. To that aim, we analyze the feedback reports that our team of Teaching Assistants gave to the 230 students of our 2016-2017 software testing course at Delft University of Technology. We also survey 84 students and seven of our teaching assistants on their perceptions. Our results help educators not only to propose pragmatic software testing courses in their faculties, but also bring understanding on the challenges that software testing students face when taking software testing courses.

References

[1]

Maurício Aniche, Felienne Hermans, and Arie van Deursen. 2018. Pragmatic Software Testing Education: Appendix. (2018).

[2]

Tara Astigarraga, Eli M Dow, Christina Lara, Richard Prewitt, and Maria R Ward. 2010. The emerging role of software testing in curricula. In Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments, 2010 IEEE. IEEE, 1--26.

[3]

Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering, Vol. 20, 4 (2015), 1052--1094.

Digital Library

[4]

Antonia Bertolino. 2007. Software testing research: Achievements, challenges, dreams. In 2007 Future of Software Engineering. IEEE Computer Society, 85--103.

Digital Library

[5]

Rex Black, Erik Van Veenendaal, and Dorothy Graham. 2012. Foundations of software testing: ISTQB certification .Cengage Learning.

Digital Library

[6]

FT Chan, TH Tse, WH Tang, and TY Chen. 2005. Software testing education and training in Hong Kong. In Quality Software, 2005.(QSIC 2005). Fifth International Conference on. IEEE, 313--316.

Digital Library

[7]

John Joseph Chilenski and Steven P Miller. 1994. Applicability of modified condition/decision coverage to software testing. Software Engineering Journal, Vol. 9, 5 (1994), 193--200.

[8]

Peter J Clarke, Debra Davis, Tariq M King, Jairo Pava, and Edward L Jones. 2014. Integrating testing into software engineering courses supported by a collaborative learning environment. ACM Transactions on Computing Education (TOCE), Vol. 14, 3 (2014), 18.

Digital Library

[9]

Stephen H Edwards. 2003. Rethinking computer science education from a test-first perspective. In Companion of the 18th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. ACM, 148--155.

Digital Library

[10]

Stephen H Edwards. 2004. Using software testing to move students from trial-and-error to reflection-in-action. ACM SIGCSE Bulletin, Vol. 36, 1 (2004), 26--30.

Digital Library

[11]

Sebastian Elbaum, Suzette Person, Jon Dokulil, and Matt Jorde. 2007. Bug hunt: Making early software testing lessons engaging and affordable. In Proceedings of the 29th international conference on Software Engineering. IEEE Computer Society, 688--697.

Digital Library

[12]

Facebook. {n. d.}. Building and Testing at Facebook. https://www.facebook.com/notes/facebook-engineering/building-and-testing-at-facebook/10151004157328920/. ({n. d.}). Last visited in October, 2017.

[13]

Steve Freeman and Nat Pryce. 2009. Growing object-oriented software, guided by tests .Pearson Education.

Digital Library

[14]

Vahid Garousi and Aditya Mathur. 2010. Current state of the software testing education in north american academia and some recommendations for the new educators. In Software Engineering Education and Training (CSEE&T), 2010 23rd IEEE Conference on. IEEE, 89--96.

Digital Library

[15]

Vahid Garousi and Junji Zhi. 2013. A survey of software testing practices in Canada. Journal of Systems and Software, Vol. 86, 5 (2013), 1354--1376.

Digital Library

[16]

Judith L Gersting. 1994. A software engineering "frosting" on a traditional CS-1 course. In ACM SIGCSE Bulletin, Vol. 26. ACM, 233--237.

Digital Library

[17]

Michael H Goldwasser. 2002. A gimmick to integrate software testing throughout the curriculum. In ACM SIGCSE Bulletin, Vol. 34. ACM, 271--275.

Digital Library

[18]

Misko Hevery. 2008. Testability explorer: using byte-code analysis to engineer lasting social changes in an organization's software development process. In Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications. ACM, 747--748.

Digital Library

[19]

Thomas B Hilburn. 1996. Software engineering-from the beginning. In Software Engineering Education, 1996. Proceedings., Ninth Conference on. IEEE, 29--39.

Digital Library

[20]

Thomas B Hilburn and Massood Townhidnejad. 2000. Software quality: a curriculum postscript?. In ACM SIGCSE Bulletin, Vol. 32. ACM, 167--171.

Digital Library

[21]

Ursula Jackson, Bill Z Manaris, and Renée A McCauley. 1997. Strategies for effective integration of software engineering concepts and techniques into the undergraduate computer science curriculum. In ACM SIGCSE Bulletin, Vol. 29. ACM, 360--364.

Digital Library

[22]

David Janzen and Hossein Saiedian. 2008. Test-driven learning in early programming courses. ACM SIGCSE Bulletin, Vol. 40. ACM, 532--536.

Digital Library

[23]

Edward L Jones. 2001 a. An experiential approach to incorporating software testing into the computer science curriculum. In Frontiers in Education Conference, 2001. 31st Annual, Vol. 2. IEEE, F3D--7.

Digital Library

[24]

Edward L Jones. 2001 b. Integrating testing into the curriculum-arsenic in small doses. ACM SIGCSE Bulletin, Vol. 33, 1 (2001), 337--341.

Digital Library

[25]

Jeff Langr, Andy Hunt, and Dave Thomas. 2015. Pragmatic unit testing in Java 8 with JUnit .The Pragmatic Bookshelf.

Digital Library

[26]

Otávio Augusto Lazzarini Lemos, Fábio Fagundes Silveira, Fabiano Cutigi Ferrari, and Alessandro Garcia. 2017. The impact of Software Testing education on code reliability: An empirical assessment. Journal of Systems and Software (2017).

[27]

Will Marrero and Amber Settle. 2005. Testing first: emphasizing testing in early programming courses. In ACM SIGCSE Bulletin, Vol. 37. ACM, 4--8.

Digital Library

[28]

Robert C Martin. 2002. Agile software development: principles, patterns, and practices .Prentice Hall.

[29]

Robert C Martin. 2017. Clean architecture: a craftsman's guide to software structure and design .Prentice Hall Press.

Digital Library

[30]

Scott Matteson. {n. d.}. Report: Software failure caused 1.7 trillion in financial losses in 2017. https://www.techrepublic.com/article/report-software-failure-caused-1--7-trillion-in-financial-losses-in-2017/. ({n. d.}).

[31]

Renée McCauley and Ursula Jackson. 1999. Teaching software engineering early: experiences and results. ACM SIGCSE Bulletin, Vol. 31, 2 (1999), 86--91.

Digital Library

[32]

Gerard Meszaros. 2007. xUnit test patterns: Refactoring test code .Pearson Education.

Digital Library

[33]

G. Miller. {n. d.}. A Scientist's Nightmare: Software Problem Leads to Five Retractions. http://science.sciencemag.org/content/314/5807/1856.full. ({n. d.}). Last visited in October, 2017.

[34]

SP Ng, Tafline Murnane, Karl Reed, D Grant, and TY Chen. 2004. A preliminary survey on software testing practices in Australia. In Software Engineering Conference, 2004. Proceedings. 2004 Australian. IEEE, 116--125.

Digital Library

[35]

Alan Page, Ken Johnston, and Bj Rollison. 2008. How we test software at Microsoft .Microsoft Press.

Digital Library

[36]

James Robergé and Candice Suriano. 1994. Using laboratories to teach software engineering principles in the introductory computer science curriculum. In ACM SIGCSE Bulletin, Vol. 26. ACM, 106--110.

Digital Library

[37]

Johnny Salda na. 2015. The coding manual for qualitative researchers .Sage.

[38]

Jaime Spacco and William Pugh. 2006. Helping students appreciate test-driven development (TDD). In Companion to the 21st ACM SIGPLAN symposium on Object-oriented programming systems, languages, and applications. ACM, 907--913.

Digital Library

[39]

Muhammad Dhiauddin Mohamed Suffian, Suhaimi Ibrahim, and Mohamed Redzuan Abdullah. 2014. A proposal of postgraduate programme for software testing specialization. In Software Engineering Conference (MySEC), 2014 8th Malaysian. IEEE, 342--347.

[40]

Joseph Timoney, Stephen Brown, and Deshi Ye. 2008. Experiences in software testing education: some observations from an international cooperation. In Young Computer Scientists, 2008. ICYCS 2008. The 9th International Conference for. IEEE, 2686--2691.

Digital Library

[41]

James A Whittaker, Jason Arbon, and Jeff Carollo. 2012. How Google tests software .Addison-Wesley.

Digital Library

[42]

Eric Wong. 2012. Improving the state of undergraduate software testing education. In American Society for Engineering Education. American Society for Engineering Education.

Cited By

Land KVogel-Heuser B(2024)Investigating the graphical IEC 61131-3 language impact on test case design and evaluation of mechatronic apprenticesat - Automatisierungstechnik10.1515/auto-2023-016272:3(176-188)Online publication date: 29-Feb-2024
https://doi.org/10.1515/auto-2023-0162
Georg Barth MThorgeirsson SSu Z(2024)A Direct Manipulation Programming Environment for Teaching Introductory and Advanced Software TestingProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699564(1-11)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699564
Fasolino ATramontana PCoppola RArdito LFraser GLeotta M(2024)Test Smells Learning by a Gamification ApproachProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685687(30-33)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685687
Show More Cited By

Index Terms

Pragmatic Software Testing Education
1. Applied computing
  1. Education
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation

Recommendations

Software Testing in Introductory Programming Courses: A Systematic Mapping Study
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Traditionally, students learn about software testing during intermediate or advanced computing courses. However, it is widely advocated that testing should be addressed beginning in introductory programming courses. In this context, testing practices ...
Testing Education: A Survey on a Global Scale
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software Engineering

[Background]: The software industry has a high demand for professionals in the software quality area. However, computing students, in general, leave the university with little or no knowledge in software testing. [Aim] Although previous studies have ...
Gamifying a Software Testing Course with Code Defenders
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Software testing is an essential skill for software developers, but it is challenging to get students engaged in this activity. The Code Defenders game addresses this problem by letting students compete over code under test by either introducing faults ("...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

February 2019

1364 pages

ISBN:9781450358903

DOI:10.1145/3287324

General Chairs:
Elizabeth K. Hawthorne
Union County College, USA
,
Manuel A. Pérez-Quiñones
University of North Carolina at Charlotte, USA
,
Program Chairs:
Sarah Heckman
North Carolina State University, USA
,
Jian Zhang
Texas Woman's University, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '19

Sponsor:

SIGCSE

SIGCSE '19: The 50th ACM Technical Symposium on Computer Science Education

February 27 - March 2, 2019

MN, Minneapolis, USA

Acceptance Rates

SIGCSE '19 Paper Acceptance Rate 169 of 526 submissions, 32%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025

Sponsor:
sigcse

The 56th ACM Technical Symposium on Computer Science Education

February 26 - March 1, 2025

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
836
Total Downloads

Downloads (Last 12 months)129
Downloads (Last 6 weeks)12

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Land KVogel-Heuser B(2024)Investigating the graphical IEC 61131-3 language impact on test case design and evaluation of mechatronic apprenticesat - Automatisierungstechnik10.1515/auto-2023-016272:3(176-188)Online publication date: 29-Feb-2024
https://doi.org/10.1515/auto-2023-0162
Georg Barth MThorgeirsson SSu Z(2024)A Direct Manipulation Programming Environment for Teaching Introductory and Advanced Software TestingProceedings of the 24th Koli Calling International Conference on Computing Education Research10.1145/3699538.3699564(1-11)Online publication date: 12-Nov-2024
https://dl.acm.org/doi/10.1145/3699538.3699564
Fasolino ATramontana PCoppola RArdito LFraser GLeotta M(2024)Test Smells Learning by a Gamification ApproachProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685687(30-33)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685687
Mansur RShaffer CEdwards SDorodchi MZhange MCooper S(2024)Mutating Matters: Analyzing the Influence of Mutation Testing in Programming CoursesProceedings of the 2024 on ACM Virtual Global Computing Education Conference V. 110.1145/3649165.3690110(151-157)Online publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1145/3649165.3690110
Sterk AWessel MHooten EZaidman ASaadatmand MLonetti FBudnik CLi JGuerriero A(2024)Running a Red Light: An Investigation into Why Software Engineers (Occasionally) Ignore Coverage ChecksProceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024)10.1145/3644032.3644444(12-22)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3644032.3644444
Aludhilu HSutinen E(2024)Bridging the Gap: Addressing Software Testing Challenges in Namibian Startups through a Tailored Training ApproachProceedings of the 7th ACM/IEEE International Workshop on Software-intensive Business10.1145/3643690.3648244(79-86)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.1145/3643690.3648244
Pawagi MKumar V(2024)Probeable Problems for Beginner-level Programming-with-AI ContestsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671108(166-176)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671108
Bai GJiang ZPrice TStolee K(2024)Evaluating the Effectiveness of a Testing Checklist Intervention in CS2: An Quasi-experimental Replication StudyProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671102(55-64)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671102
Nadeem AStephenson BStone JBattestilli LRebelsky SShoop L(2024)Cybersecurity as a Crosscutting Concept Across an Undergrad Computer Science Curriculum: An Experience ReportProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630821(916-922)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630821
Haldar SPierce MCapretz L(2024)Factors Influencing the Performance of Students in Software Automated Test Tools Course2024 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW60967.2024.00064(331-339)Online publication date: 27-May-2024
https://doi.org/10.1109/ICSTW60967.2024.00064
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents