[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2542050.2542074acmotherconferencesArticle/Chapter ViewAbstractPublication PagessoictConference Proceedingsconference-collections
research-article

Constructing test cases for n-wise testing from tree-based test models

Published: 05 December 2013 Publication History

Abstract

In our previous work [17], we proposed a model-based combinatorial testing method, called FOT. It provides a technique to design test-models for combinatorial testing based on extended logic trees. In this paper, we introduce pair-wise testing (and by extension, n-wise testing, where n = 1, 2, ...) to FOT, by developing a technique to construct a test-suite of n-wise strategies from the test models in FOT. We take a "transformation approach" to realize this technique. To construct test suites, this approach first transforms test-models in FOT, represented as extended logic trees, to those in the formats which the existing n-wise testing tools (such as PICT [9], ACTS [30], CIT-BACH [31], etc.) accept to input, and then applies transformed test-models to any of these tools. In this transformation approach, an algorithm, called "flattening algorithm", plays a key role. We prove the correctness of the algorithm, and implement the algorithm to automate such test-suite constructions, providing a tool called FOT-nw (FOT with n-wise). Further, to show the effectiveness of the technique, we conduct a case study, where we apply FOT-nw to design test models and automatically construct test suites of n-wise strategies for an embedded system of stationary services for real-use in industry.

References

[1]
P. Ammann, J. Offutt, and W. Xu. Coverage criteria for state based specifications. In Formal Methods and Testing, volume 4949 of LNCS, pages 118--156. Springer, 2008.
[2]
C. Artho, A. Biere, M. Hagiya, E. Platon, M. Seidl, Y. Tanabe, and M. Yamamoto. Modbat: A model-based API tester for event-driven systems. In Proc. 9th Haifa Verification Conference (HVC 2013), volume 8244 of LNCS, pages 112--128, Haifa, Israel, 2013. Springer.
[3]
M. N. Borazjany, L. Yu, Y. Lei, R. Kacker, and R. Kuhn. Combinatorial testing of ACTS: A case study. In Proc. of ICST'12, pages 591--600, 2012.
[4]
R. C. Bryce and C. J. Colbourn. Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information & Software Technology, 48(10): 960--970, 2006.
[5]
D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton. The AETG system: An approach to testing based on combinatorial design. IEEE Trans. Software Eng., 23(7): 437--444, 1997.
[6]
D. M. Cohen, S. R. Dalal, J. Parelius, and G. C. Patton. The combinatorial design approach to automatic test generation. IEEE Software, 13(5): 83--88, 1996.
[7]
M. B. Cohen, M. B. Dwyer, and J. Shi. Interaction testing of highly-configurable systems in the presence of constraints. In Proc. of ISSTA'07, pages 129--139, 2007.
[8]
M. B. Cohen, M. B. Dwyer, and J. Shi. Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Trans. Software Eng., 34(5): 633--650, 2008.
[9]
J. Czerwonka. Pairwise testing in the real world. practical extensions to test case generators. Microsoft Corporation, Software Testing Technical Articles, 2008.
[10]
Y. L. D. Richard Kuhn, Raghu N. Kacker. Practical Combinatorial Testing. Technical Report NIST/SP-800-142, National Institute of Standards and Technology, 2010.
[11]
C. Ericson. Fault tree analysis - a history. In The 17th International Systems Safety Conference, 1999.
[12]
M. Grindal and J. Offutt. Input parameter modeling for combination strategies. In Proc. of IASTED'07 on Software Engineering, pages 255--260. ACTA Press, 2007.
[13]
M. Grindal, J. Offutt, and J. Mellin. Handling constraints in the input space when using combination strategies for software testing. Technical report, School of Humanities and Informatics, University of Skovde, 2006.
[14]
M. Grochtmann. Test case design using classification trees. In Proc. of The International Conference on Software Testing Analysis '94, 1994.
[15]
M. Grochtmann and J. Wegener. Test case design using classification trees and the classification-tree editor cte. In Proc. of QW'95, 1995.
[16]
K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-oriented domain analysis (FODA) feasibility study. Technical Report CMU/SEI-90-TR-21, Carnegie-Mellon University Software Engineering Institute, 1990.
[17]
T. Kitamura, T. B. N. Do, H. Ohsaki, L. Fang, and S. Yatabe. Test-case design by feature trees. In Proc. of ISoLA'12 (1), volume 7609 of LNCS, pages 458--473. Springer, 2012.
[18]
P. Kruse and M. Luniak. Automated test case generation using classification trees. Software Quality Professional, 13(1), 2010.
[19]
D. R. Kuhn, D. R. Wallace, and A. M. Gallo. Software fault interactions and implications for software testing. IEEE Trans. Software Eng., 30(6): 418--421, 2004.
[20]
S. Mauw and M. Oostdijk. Foundations of attack trees. In ICISC, pages 186--198, 2005.
[21]
J. D. McCaffrey and J. Czerwonka. An empirical study of the effectiveness of pairwise testing. In Proc. of SERP'09, pages 186--191, 2009.
[22]
T. Nanba, T. Tsuchiya, and T. Kikuno. Using satisfiability solving for pairwise testing in the presence of constraints. IEICE Transactions, 95-A(9): 1501--1505, 2012.
[23]
OSEK/VDX operating system specification 2.2.3 http://www.osek-vdx.org/, 2005.
[24]
S. Oster, F. Markert, and P. Ritter. Automated incremental pairwise testing of software product lines. In Proc. of SPLC'10, pages 196--210, 2010.
[25]
S. Oster, M. Zink, M. Lochau, and M. Grechanik. Pairwise feature-interaction testing for spls: potentials and limitations. In Proc. of SPLC'11 (2), page 6, 2011.
[26]
J. Petke, S. Yoo, M. B. Cohen, and M. Harman. Efficiency and early fault detection with lower and higher strength combinatorial interaction testing. In Proc. of ESEC/FSE '13, 2013.
[27]
J. R. Quinlan. Induction of decision trees. Machine Learning, 1(1): 81--106, 1986.
[28]
A. van Lamsweerde. Goal-oriented requirements engineering: A guided tour. In Proc. of RE, page 249, 2001.
[29]
D. B. West. Introduction to Graph Theory. Pearson, 2000.
[30]
ACTS, available from: http://csrc.nist.gov/groups/SNS/acts/index.html.
[31]
CIT-BACH, available from: http://www-ise4.ist.osaka-u.ac.jp/t-tutiya/CIT/.

Cited By

View all
  • (2018)Software TestingSoftware Quality: Concepts and Practice10.1002/9781119134527.ch14(255-317)Online publication date: 6-Apr-2018
  • (2017)Combinatorial Methods for Modelling Composed Software Systems2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2017.43(229-238)Online publication date: Mar-2017
  • (2017)A novel approach for combinatorial test case generation using multi objective optimization2017 7th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE.2017.8167914(411-418)Online publication date: Oct-2017
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
SoICT '13: Proceedings of the 4th Symposium on Information and Communication Technology
December 2013
345 pages
ISBN:9781450324540
DOI:10.1145/2542050
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

  • SOICT: School of Information and Communication Technology - HUST
  • NAFOSTED: The National Foundation for Science and Technology Development
  • ACM Vietnam Chapter: ACM Vietnam Chapter
  • Danang Univ. of Technol.: Danang University of Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2013

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automated test case generation
  2. black box testing
  3. n-wise testing

Qualifiers

  • Research-article

Conference

SoICT '13
Sponsor:
  • SOICT
  • NAFOSTED
  • ACM Vietnam Chapter
  • Danang Univ. of Technol.

Acceptance Rates

SoICT '13 Paper Acceptance Rate 40 of 80 submissions, 50%;
Overall Acceptance Rate 147 of 318 submissions, 46%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)1
Reflects downloads up to 07 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2018)Software TestingSoftware Quality: Concepts and Practice10.1002/9781119134527.ch14(255-317)Online publication date: 6-Apr-2018
  • (2017)Combinatorial Methods for Modelling Composed Software Systems2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW.2017.43(229-238)Online publication date: Mar-2017
  • (2017)A novel approach for combinatorial test case generation using multi objective optimization2017 7th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE.2017.8167914(411-418)Online publication date: Oct-2017
  • (2015)Combinatorial Testing for Tree-Structured Test Models with ConstraintsProceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security10.1109/QRS.2015.29(141-150)Online publication date: 3-Aug-2015
  • (2015)Priority Integration for Weighted Combinatorial TestingProceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 0210.1109/COMPSAC.2015.113(242-247)Online publication date: 1-Jul-2015
  • (2014)Design of Prioritized N-Wise TestingProceedings of the 26th IFIP WG 6.1 International Conference on Testing Software and Systems - Volume 876310.1007/978-3-662-44857-1_14(186-191)Online publication date: 23-Sep-2014

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media