A Comparative Study on Combinatorial and Random Testing for Highly Configurable Systems

1. Arcuri, A., Briand, L.C.: Formal analysis of the probability of interaction fault detection using random testing. IEEE Trans. Softw. Eng. 38(5), 1088–1099 (2012)

   Article  Google Scholar 

2. Chakraborty, S., Fremont, D.J., Meel, K.S., Seshia, S.A., Vardi, M.Y.: On parallel scalable uniform sat witness generation. In: Baier, C., Tinelli, C. (eds.) TACAS 2015. LNCS, vol. 9035, pp. 304–319. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46681-0_25

   Chapter  Google Scholar 

3. Chen, T.Y., Leung, H., Mak, I.K.: Adaptive random testing. In: Maher, M.J. (ed.) ASIAN 2004. LNCS, vol. 3321, pp. 320–329. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30502-6_23

   Chapter  Google Scholar 

4. Dutra, R., Laeufer, K., Bachrach, J., Sen, K.: Efficient sampling of SAT solutions for testing. In: Proceedings of ICSE 2018, pp. 549–559 (2018)

   Google Scholar 

5. Ermon, S., Gomes, C., Selman, B.: Uniform solution sampling using a constraint solver as an oracle. In: Proceedings of UAI 2012, pp. 255–264 (2012)

   Google Scholar 

6. Gargantini, A., Radavelli, M.: Migrating combinatorial interaction test modeling and generation to the web. In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 308–317, April 2018. https://doi.org/10.1109/ICSTW.2018.00066

7. Garvin, B.J., Cohen, M.B., Dwyer, M.B.: Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Softw. Eng. 16(1), 61–102 (2011)

   Article  Google Scholar 

8. Hirasaki, Y., Kojima, H., Tsuchiya, T.: Applying random testing to constrained interaction testing. In: Proceedings of SEKE 2013, pp. 193–198 (2014)

   Google Scholar 

9. Johansen, M.F., Haugen, O., Fleurey, F.: An algorithm for generating t-wise covering arrays from large feature models. In: Proceedings of SPLC 2012, pp. 46–55 (2012)

   Google Scholar 

10. Kuhn, D.R., Bryce, R., Duan, F., Ghandehari, L.S., Lei, Y., Kacker, R.N.: Chapter one - combinatorial testing: theory and practice. In: Advances in Computers, vol. 99, pp. 1–66. Elsevier (2015)

    Google Scholar 

11. Kuhn, D.R., Kacker, R.N., Lei, Y.: Introduction to Combinatorial Testing. CRC Press, Boca Raton (2013)

    MATH  Google Scholar 

12. Lin, J., Cai, S., Luo, C., Lin, Q., Zhang, H.: Towards more efficient meta-heuristic algorithms for combinatorial test generation. In: Proceedings of ESEC/FSE 2019, pp. 212–222 (2019)

    Google Scholar 

13. Lynce, I., Silva, J.P.M.: On computing minimum unsatisfiable cores. In: Proceedings of SAT 2004 (2004)

    Google Scholar 

14. Nanba, T., Tsuchiya, T., Kikuno, T.: Using satisfiability solving for pairwise testing in the presence of constraints. IEICE Trans. 95-A(9), 1501–1505 (2012)

    Google Scholar 

15. Oh, J., Batory, D.S., Myers, M., Siegmund, N.: Finding near-optimal configurations in product lines by random sampling. In: Proceedings of ESEC/FSE 2017, pp. 61–71 (2017)

    Google Scholar 

16. Wu, H., Nie, C., Petke, J., Jia, Y., Harman, M.: An empirical comparison of combinatorial testing, random testing and adaptive random testing. IEEE Trans. Softw. Eng. (2018)

    Google Scholar 

17. Yamada, A., Biere, A., Artho, C., Kitamura, T., Choi, E.: Greedy combinatorial test case generation using unsatisfiable cores. In: Proceedings of ASE 2016, pp. 614–624 (2016)

    Google Scholar 

18. Yu, L., Lei, Y., Kacker, R.N., Kuhn, D.R.: ACTS: a combinatorial test generation tool. In: Proceedings of ICST 2013, pp. 370–375 (2013)

    Google Scholar 

19. Yu, L., Duan, F., Lei, Y., Kacker, R., Kuhn, D.R.: Combinatorial test generation for software product lines using minimum invalid tuples. In: Proceedings of HASE 2014, pp. 65–72 (2014)

    Google Scholar 

Download references