article

Modeling requirements for combinatorial software testing

Authors:

C. Lott,

A. Jain,

S. DalalAuthors Info & Claims

ACM SIGSOFT Software Engineering Notes, Volume 30, Issue 4

Pages 1 - 7

https://doi.org/10.1145/1082983.1083281

Published: 15 May 2005 Publication History

Get Access

Abstract

The combinatorial approach to software testing uses models to generate a minimal number of test inputs so that selected combinations of input values are covered. The most common coverage criteria is two-way, or pairwise coverage of value combinations, though for higher confidence three-way or higher coverage may be required. This paper presents example system requirements and corresponding models for applying the combinatorial approach to those requirements. These examples are intended to serve as a tutorial for applying the combinatorial approach to software testing. Although this paper focuses on pairwise coverage, the discussion is equally valid when higher coverage criteria such as three-way (triples) are used. We use terminology and modeling notation from the AETG¹ system to provide concrete examples.

References

[1]

Renée C. Bryce, Charles Colbourn, and Myra Cohen. A framework of greedy methods for constructing interaction test suites. In Proceedings of the 2005 International Conference on Software Engineering. ACM Press, May 2005.

Digital Library

Google Scholar

[2]

David M. Cohen, Siddhartha R. Dalal, Michael L. Fredman, and Gardner C. Patton. The AETG system: An approach to testing based on combinatorial design. IEEE Transactions on Software Engineering, 23(7):437--444, July 1997.

Digital Library

Google Scholar

[3]

S. Dalal, A. Jain, C. Lott, G. Patton, N. Karunanith, J. M. Leaton, and B. M. Horowitz. Model-based testing in practice. In Proceedings of the 21st International Conference on Software Engineering, pages 285--294. ACM Press, May 1999.

Digital Library

Google Scholar

[4]

Siddhartha R. Dalal and Colin L. Mallows. Factor-covering designs for testing software. Technometrics, 40(3):234--243, August 1998.

Digital Library

Google Scholar

[5]

I. S. Dunietz, W. K. Ehrlich, B. D. Szablak, C. L. Mallows, and A. Iannino. Applying design of experiments to software testing. In Proceedings of the Nineteenth International Conference on Software Engineering, pages 205--215. ACM Press, May 1997.

Digital Library

Google Scholar

[6]

IBM. Combinatorial Test Services. http://www.alphaworks.ibm.com/tech/cts, 2004.

Google Scholar

Cited By

View all

Zhang BLi JFan ZWang Y(2023)An Input Space Modeling Method for Software Testing Considering Timing2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00036(614-621)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00036
Tatale SChandra Prakash V(2022)Combinatorial test case generation from sequence diagram using optimization algorithmsInternational Journal of System Assurance Engineering and Management10.1007/s13198-021-01579-w13:S1(642-657)Online publication date: 4-Jan-2022
https://doi.org/10.1007/s13198-021-01579-w
Kacker RKuhn DLei YSimos D(2021)Factorials Experiments, Covering Arrays, and Combinatorial TestingMathematics in Computer Science10.1007/s11786-021-00502-715:4(715-739)Online publication date: 12-Apr-2021
https://doi.org/10.1007/s11786-021-00502-7
Show More Cited By

Index Terms

Modeling requirements for combinatorial software testing
1. Computing methodologies
  1. Modeling and simulation
    1. Model development and analysis
2. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Combinatorial testing: learnings from our experience

Combinatorial testing methods address generation of test cases for problems involving multiple parameters and combinations. The Orthogonal Array Based Testing Strategy (OATS) is one such combinatorial testing method, a systematic, statistical way of ...
Modeling requirements for combinatorial software testing
A-MOST '05: Proceedings of the 1st international workshop on Advances in model-based testing

The combinatorial approach to software testing uses models to generate a minimal number of test inputs so that selected combinations of input values are covered. The most common coverage criteria is two-way, or pairwise coverage of value combinations, ...
Combinatorial Software Testing

Combinatorial testing can detect hard-to-find software faults more efficiently than manual test case selection methods.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGSOFT Software Engineering Notes

ACM SIGSOFT Software Engineering Notes Volume 30, Issue 4

July 2005

1514 pages

ISSN:0163-5948

DOI:10.1145/1082983

Issue’s Table of Contents

A-MOST '05: Proceedings of the 1st international workshop on Advances in model-based testing
May 2005
136 pages
ISBN:1595931155
DOI:10.1145/1083274

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2005

Published in SIGSOFT Volume 30, Issue 4

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
815
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang BLi JFan ZWang Y(2023)An Input Space Modeling Method for Software Testing Considering Timing2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C60940.2023.00036(614-621)Online publication date: 22-Oct-2023
https://doi.org/10.1109/QRS-C60940.2023.00036
Tatale SChandra Prakash V(2022)Combinatorial test case generation from sequence diagram using optimization algorithmsInternational Journal of System Assurance Engineering and Management10.1007/s13198-021-01579-w13:S1(642-657)Online publication date: 4-Jan-2022
https://doi.org/10.1007/s13198-021-01579-w
Kacker RKuhn DLei YSimos D(2021)Factorials Experiments, Covering Arrays, and Combinatorial TestingMathematics in Computer Science10.1007/s11786-021-00502-715:4(715-739)Online publication date: 12-Apr-2021
https://doi.org/10.1007/s11786-021-00502-7
Tzoref-Brill R(2018)Advances in Combinatorial Testing10.1016/bs.adcom.2017.12.002Online publication date: 2018
https://doi.org/10.1016/bs.adcom.2017.12.002
Tzoref-Brill RMaoz SUchitel SOrso ARobillard M(2017)Syntactic and semantic differencing for combinatorial models of test designsProceedings of the 39th International Conference on Software Engineering10.1109/ICSE.2017.63(621-631)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.1109/ICSE.2017.63
Borck HKline PShackleton HGohde JJohnston SAlexander PCarpenter T(2017)100 Years of Software - Adapting Cyber-Physical Systems to the Changing WorldSoftware Engineering for Resilient Systems10.1007/978-3-319-65948-0_9(133-148)Online publication date: 11-Aug-2017
https://doi.org/10.1007/978-3-319-65948-0_9
Dalal SJain AKantor P(2015)Creating configurations for testing radiation portal algorithms using factor covering combinatorial designs2015 IEEE International Symposium on Technologies for Homeland Security (HST)10.1109/THS.2015.7445579(1-6)Online publication date: Apr-2015
https://doi.org/10.1109/THS.2015.7445579
Mahmoud TAhmed B(2015)An efficient strategy for covering array construction with fuzzy logic-based adaptive swarm optimization for software testing useExpert Systems with Applications: An International Journal10.1016/j.eswa.2015.07.02942:22(8753-8765)Online publication date: 1-Dec-2015
https://dl.acm.org/doi/10.1016/j.eswa.2015.07.029
Zhang JZhang ZMa FZhang JZhang ZMa F(2014)Introduction to Combinatorial TestingAutomatic Generation of Combinatorial Test Data10.1007/978-3-662-45919-5_1(1-16)Online publication date: 16-Sep-2014
https://doi.org/10.1007/978-3-662-45919-5_1
Zhang JZhang ZMa FZhang JZhang ZMa F(2014)Introduction to Combinatorial TestingAutomatic Generation of Combinatorial Test Data10.1007/978-3-662-43429-1_1(1-16)Online publication date: 16-Sep-2014
https://doi.org/10.1007/978-3-662-43429-1_1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Combinatorial testing: learnings from our experience

Modeling requirements for combinatorial software testing

Combinatorial Software Testing