FRAFOL: FRAmework FOr Learning mutation testing
Pages 1846 - 1850
Abstract
Mutation testing has evolved beyond academic research, is deployed in industrial and open-source settings, and is increasingly part of universities' software engineering curricula. While many mutation testing tools exist, each with different strengths and weaknesses, integrating them into educational activities and exercises remains challenging due to the tools' complexity and the need to integrate them into a development environment. Additionally, it may be desirable to use different tools so that students can explore differences, e.g., in the types or numbers of generated mutants. Asking students to install and learn multiple tools would only compound technical complexity and likely result in unwanted differences in how and what students learn. This paper presents FRAFOL, a framework for learning mutation testing. FRAFOL provides a common environment for using different mutation testing tools in an educational setting.
References
[1]
Domenico Amalfitano, Ana C. R. Paiva, Alexis Inquel, Luís Pinto, Anna Rita Fasolino, and René Just. 2022. How do Java mutation tools differ? Commun. ACM, 65, 12 (2022), nov, 74–89. issn:0001-0782 https://doi.org/10.1145/3526099
[2]
James H Andrews, Lionel C Briand, and Yvan Labiche. 2005. Is mutation an appropriate tool for testing experiments? In Proceedings of the 27th international conference on Software engineering. 402–411.
[3]
Moritz Beller, Chu-Pan Wong, Johannes Bader, Andrew Scott, Mateusz Machalica, Satish Chandra, and Erik Meijer. 2021. What It Would Take to Use Mutation Testing in Industry—A Study at Facebook. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 268–277. https://doi.org/10.1109/ICSE-SEIP52600.2021.00036
[4]
Yiqun T Chen, Rahul Gopinath, Anita Tadakamalla, Michael D Ernst, Reid Holmes, Gordon Fraser, Paul Ammann, and René Just. 2020. Revisiting the relationship between fault detection, test adequacy criteria, and test set size. In Proceedings of the 35th IEEE/ACM international conference on automated software engineering. 237–249.
[5]
P. J. Clarke, A. A. Allen, T. M. King, E. L. Jones, and P. Natesan. [n. d.]. Using a web-based repository to integrate testing tools into programming courses. In Proceedings of the ACM International Conference Companion on Object Oriented Programming Systems Languages and Applications Companion, SPLASH ’10. 193–200. https://doi.org/10.1145/1869542.1869573
[6]
Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2016. PIT: a practical mutation testing tool for Java (demo). In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). Association for Computing Machinery, New York, NY, USA. 449–452. isbn:9781450343909 https://doi.org/10.1145/2931037.2948707
[7]
Gordon Fraser, Alessio Gambi, and José Miguel Rojas. 2020. Teaching Software Testing with the Code Defenders Testing Game: Experiences and Improvements. In 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 461–464. https://doi.org/10.1109/ICSTW50294.2020.00082
[8]
V. Garousi, M. Felderer, M. Kuhrmann, K. Herkiloğlu, and S. Eldh. 2020. Exploring the industry’s challenges in software testing: An empirical study. Journal of Software: Evolution and Process, 32, 8 (2020), https://doi.org/10.1002/smr.2251
[9]
Luca Gazzola, Daniela Micucci, and Leonardo Mariani. 2019. Automatic Software Repair: A Survey. IEEE Transactions on Software Engineering, 45, 1 (2019), 34–67. https://doi.org/10.1109/TSE.2017.2755013
[10]
Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical program repair via bytecode mutation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019). Association for Computing Machinery, New York, NY, USA. 19–30. isbn:9781450362245 https://doi.org/10.1145/3293882.3330559
[11]
René Just. 2014. The Major mutation framework: Efficient and scalable mutation analysis for Java. In Proceedings of the 2014 international symposium on software testing and analysis. 433–436.
[12]
René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 international symposium on software testing and analysis. 437–440.
[13]
René Just, Darioush Jalali, Laura Inozemtseva, Michael D. Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing? In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). Association for Computing Machinery, New York, NY, USA. 654–665. isbn:9781450330565 https://doi.org/10.1145/2635868.2635929
[14]
A. Jefferson Offutt and Roland H. Untch. 2001. Mutation 2000: Uniting the Orthogonal. Springer US, Boston, MA. 34–44. isbn:978-1-4757-5939-6 https://doi.org/10.1007/978-1-4757-5939-6_7
[15]
Goran Petrović, Marko Ivanković, Gordon Fraser, and René Just. 2021. Does mutation testing improve testing practices? In Proceedings of the 43rd International Conference on Software Engineering (ICSE ’21). IEEE Press, 910–921. isbn:9781450390859 https://doi.org/10.1109/ICSE43902.2021.00087
[16]
Goran Petrović, Marko Ivanković, Gordon Fraser, and René Just. 2021. Practical mutation testing at scale: A view from google. IEEE Transactions on Software Engineering, 48, 10 (2021), 3900–3912.
[17]
Goran Petrovic, Marko Ivankovic, Bob Kurtz, Paul Ammann, and René Just. 2018. An Industrial Application of Mutation Testing: Lessons, Challenges, and Research Directions. In 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 47–53. https://doi.org/10.1109/ICSTW.2018.00027
Index Terms
- FRAFOL: FRAmework FOr Learning mutation testing
Recommendations
Predictive mutation testing
ISSTA 2016: Proceedings of the 25th International Symposium on Software Testing and AnalysisMutation testing is a powerful methodology for evaluating test suite quality. In mutation testing, a large number of mutants are generated and executed against the test suite to check the ratio of killed mutants. Therefore, mutation testing is widely ...
Mutation testing cost reduction by clustering overlapped mutants
We defined the term conditionally-overlapped (c-overlapped) mutants.C-overlapped mutants are expected to produce the same results against a test case.Clustering c-overlapped mutants effectively reduces the cost of mutation testing.Clustering c-...
µDroid: an energy-aware mutation testing framework for Android
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software EngineeringThe rising popularity of mobile apps deployed on battery-constrained devices underlines the need for effectively evaluating their energy properties. However, currently there is a lack of testing tools for evaluating the energy properties of apps. As a ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2024
1928 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- ENACTEST Erasmus+
- National Science Foundation
Conference
ISSTA '24
Sponsor:
ISSTA '24: 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis
September 16 - 20, 2024
Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 58 of 213 submissions, 27%
Upcoming Conference
ISSTA '25
- Sponsor:
- sigsoft
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 122Total Downloads
- Downloads (Last 12 months)122
- Downloads (Last 6 weeks)33
Reflects downloads up to 15 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in