Abstract
The number of detected bugs by software test data determines the efficacy of the test data. One of the most important topics in software engineering is software mutation testing, which is used to evaluate the efficiency of software test methods. The syntactical modifications are made to the program source code to make buggy (mutated) programs, and then the resulting mutants (buggy programs) along with the original programs are executed with the test data. Mutation testing has several drawbacks, one of which is its high computational cost. Higher execution time of mutation tests is a challenging problem in the software engineering field. The major goal of this work is to reduce the time and cost of mutation testing. Mutants are inserted in each instruction of a program using typical mutation procedures and tools. Meanwhile, in a real-world program, the likelihood of a bug occurrence in the simple and non-bug-prone sections of a program is quite low. According to the 80–20 rule, 80 percent of a program's bugs are discovered in 20% of its fault-prone code. The first stage of the suggested solution uses a discretized and modified version of the Forrest optimization algorithm to identify the program's most bug-prone paths; the second stage injects mutants just in the identified bug-prone instructions and data. In the second step, the mutation operators are only injected into the identified instructions and data that are bug-prone. Studies on standard benchmark programs have shown that the proposed method reduces about 27.63% of the created mutants when compared to existing techniques. If the number of produced mutants is decreased, the cost of mutation testing will also decrease. The proposed method is independent of the platform and testing tool. The results of the experiments confirm that the use of the proposed method in each testing tool such as Mujava, Muclipse, Jester, and Jumble makes a considerable mutant reduction.
Similar content being viewed by others
Data Availability
The datasets generated during and the implemented code during the current study are available in the google.drive and can be freely accessed by the following link: https://drive.google.com/drive/folders/1eHkLdF2b-of6LqgJQaEpcyLjcfejz0Az?usp=sharing.
References
Acree AT, Budd TA, DeMillo RA, Lipton RJ, Sayward FG (1980) Mutation Analysis. School of Information and Computer Science, Georgia Institute of Technology
Aghdam ZK, Arasteh B (2017) An efficient method to generate test data for software structural testing using artificial bee colony optimization algorithm. Int J Software Eng Knowl Eng 27(6):2017
Arasteh B, Hosseini SMJ (2022) Traxtor: An Automatic Software Test Suit Generation Method Inspired by Imperialist Competitive Optimization Algorithms. J Electron Test. https://doi.org/10.1007/s10836-022-05999-9
Arasteh B, Imanzadeh P, Arasteh K et al (2022) A Source-code Aware Method for Software Mutation Testing Using Artificial Bee Colony Algorithm. J Electron Test 38:289–302. https://doi.org/10.1007/s10836-022-06008-9
Arasteh B (2019) ReDup: A software-based method for detecting soft-error using data analysis. Comp Electrical Eng 78(September 2019):89–107
Barbosa EF, Maldonado JC, Vincenzi AMR (2001) Toward the determination of sufficient mutant operators for C. Software Testing, Verification and Reliability 11(2):113–136
Bouyer A, Arasteh B, Movaghar A (2007) A New Hybrid Model Using Case-Based Reasoning and Decision Tree Methods for Improving Speedup and Accuracy. IADIS International Conference of Applied Computing
Budd TA (1980) Mutation Analysis of Program Test Data. Yale University
Chandra SSV, Sankar SS, Anand HS (2022) Smell Detection Agent Optimization Approach to Path Generation in Automated Software Testing. J Electron Test. https://doi.org/10.1007/s10836-022-06033-8
Delgado-Pérez P, Medina-Bulo I (2018) Search-based mutant selection for efficient test suite improvement: Evaluation and results. Inf Softw Technol 104(2018):130–143
Deng L, Offutt J, Ammann P, Mirzaei N (2017) Mutation operators for testing Android apps. Inf Softw Technol 81(2017):154–168
Dominguez-Jimenez JJ, Estero-Botaro A, Garcia-Dominguez A, Medina-Bulo I (2011) Evolutionary mutation testing. Inf Softw Technol 53(10):1108–1123
Fenton NE, Ohlsson N (2000) Quantitative analysis of faults and failures in a complex software system. IEEE Transact Software Eng 26(8):797–814
Ghaemi A, Arasteh B (2020) SFLA-based heuristic method to generate software structural test data. J Softw Evol Proc 32:e2228. https://doi.org/10.1002/smr.2228
Gheyi R, Ribeiro M, Souza B, Guimarães M, Fernandes L, d’Amorim M, Alves V, Teixeira L, Fonseca B (2021) Identifying method-level mutation subsumption relations using Z3. Inf Softw Technol 132:106496
Hosseini S, Arasteh B, Isazadeh A, Mohsenzadeh M, Mirzarezaee M (2021) An error-propagation aware method to reduce the software mutation cost using genetic algorithm. Data Technologies and Applications 55(1):118–148. https://doi.org/10.1108/DTA-03-2020-0073
Howden WE (1982) Weak mutation testing and completeness of test sets. IEEE Trans Software Eng 8(4):371–379
Jackson D, Woodward MR (2000) Parallel firm mutation of Java programs. Proc. First Workshop on Mutation Analysis, pp 55–61
Jia Y, Harman M (2011) An Analysis and Survey of the Development of Mutation Testing. IEEE Trans Software Eng 37(5):649–678. https://doi.org/10.1109/tse.2010.62
Keshtgar A, Arasteh B (2017) Enhancing Software Reliability against Soft-Error using Minimum Redundancy on Critical Data. https://doi.org/10.5815/ijcnis.2017.05.03
King KN, Offutt AJ (1991) A Fortran language system for mutation-based software testing. Software: Practice and Experience 21(7):685–718
Kintis M, Papadakis M, Malevris N (2010) Evaluating mutation testing alternatives: a collateral experiment. Proceedings of the 17th Asia-Pacific Software Engineering Conference (APSEC)
Kurtz B, Ammann P, Delamaro ME, Offutt J, Deng L (2014) Mutant subsumption graphs. 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
Kurtz B, Ammann P, Offutt J (2015) Static analysis of mutant subsumption. IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
Ma YS, Offutt J, Kwon YR (2006) MuJava: A Mutation System for Java. In 28th International Conference on Software Engineering (ICSE ’06)
Malevris N, Yates D (2006) The collateral coverage of data flow criteria when branch testing. Inf Softw Technol 48(8):676–686
Manizheh G, Mohammad-Reza F (2014) Forest Optimization Algorithm. Expert Syst Appl 41(15):6676–6687
Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C (1996) An experimental determination of sufficient mutant operators. ACM Trans Softw Eng Methodol 5(2):99–118
Offutt AJ, Rothermel G, Zapf C (1993) An experimental evaluation of selective mutation. Proceedings of the 15th International Conference on Software Engineering, ICSE ’93. IEEE Computer Society Press, Los Alamitos, CA
Papadakis M, Malevris N (2010) An empirical evaluation of the first and second order mutation testing strategies. 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW)
Sankar SS, Chandra SS (2020a) A Structural Testing Model Using SDA Algorithm. Lect Notes Comput Sci 405–412. https://doi.org/10.1007/978-3-030-53956-6_36
Sankar SS, Chandra SS (2020b) An Ant Colony Optimization Algorithm Based Automated Generation of Software Test Cases. Lect Notes Comput Sci 231–239. https://doi.org/10.1007/978-3-030-53956-6_21
Wei C, Yao X, Gong D, Liu H (2021) Spectral clustering based mutant reduction for mutation testing. Inf Softw Technol 132(2021):106502
Wong WE (1993) On mutation and data flow. Ph.D. dissertation, Purdue University
Yao X, Zhang G, Pan F, Gong D, Wei C (2020) Orderly Generation of Test Data via Sorting Mutant Branches Based on Their Dominance Degrees for Weak Mutation Testing. IEEE Trans Software Eng 48(4):1169–1184. https://doi.org/10.1109/tse.2020.3014960
Zhang L, Hou SS, Hu JJ, Xie T, Mei H (2010) Is operator-based mutant selection superior to random mutant selection? Proceedings of the 32Nd ACM/IEEE International Conference on Software Engineering
Zhang L, Gligoric M, Marinov D, Khurshid S (2013) Operator-based and random mutant selection: better together. Automated Software Engineering (ASE). IEEE/ACM 28th International Conference
Author information
Authors and Affiliations
Contributions
All authors contributed to the study's conception and design. The data collection, and analysis were performed by Bahman Arasteh. Experiments have been performed by Bahman Arasteh. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of Interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Responsible Editor: Y. K. Malaiya
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Arasteh, B., Gharehchopogh, F.S., Gunes, P. et al. A Novel Metaheuristic Based Method for Software Mutation Test Using the Discretized and Modified Forrest Optimization Algorithm. J Electron Test 39, 347–370 (2023). https://doi.org/10.1007/s10836-023-06070-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10836-023-06070-x