More Web Proxy on the site http://driver.im/

Article

Test prioritization for pairwise interaction coverage

Authors:

Renée C. Bryce,

Charles J. ColbournAuthors Info & Claims

A-MOST '05: Proceedings of the 1st international workshop on Advances in model-based testing

Pages 1 - 7

https://doi.org/10.1145/1083274.1083275

Published: 15 May 2005 Publication History

Abstract

Interaction testing is widely used in screening for faults. In software testing, it provides a natural mechanism for testing systems to be deployed on a variety of hardware and software configurations. Several algorithms published in the literature are used as tools to automatically generate these test suites; AETG is a well known example of a family of greedy algorithms that generate one test at a time. In many applications where interaction testing is needed, the entire test suite is not run as a result of time or cost constraints. In these situations, it is essential to prioritize the tests. Here we adapt a "one-test-at-a-time" greedy method to take importance of pairs into account. The method can be used to generate a set of tests in order, so that when run to completion all pairwise interactions are tested, but when terminated after any intermediate number of tests, those deemed most important are tested. Computational results on the method are reported.

References

[1]

R. C. Bryce, C. J. Colbourn, and M. B. Cohen. A framework of greedy methods for constructing interaction test suites. In Proc. 27th International Conference on Software Engineering (ICSE2005), page to appear, May 2005.

Digital Library

[2]

R. F. Berry. Computer bench mark evaluation and design of experiments a case study. In Proc. IEEE Wireless Communications Networking Conference (WCNC03), 41(10):1279--1289, 1992.

Digital Library

[3]

T. Berling and P. Runeson. Efficient Evaluation of Multifactor Dependent System Performance Using Fractional Factorial Design. IEEE Transactions on Software Engineering, 29(9):769--781, 2003.

Digital Library

[4]

K. Burr and W. Young. Combinatorial test techniques: Table-based automation, test generation, and code coverage. Proceedings of the Intl. Conf. on Software Testing Analysis and Review, pages 503--513, October 1998.

[5]

C. Cheng, A. Dumitrescu, and P. Schroeder. Generating small combinatorial test suites to cover input-output relationships. Proceedings of the Third International Conference on Quality Software (QSIC '03), pages 76--82, 2003.

Digital Library

[6]

D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. C. Patton. The AETG system: an approach to testing based on combinatorial design. IEEE Transactions on Software Engineering, 23(7):437--44, October 1997.

Digital Library

[7]

D. M. Cohen, S. R. Dalal, M. L. Fredman, and G. Patton. Method and system for automatically generating efficient test cases for systems having interacting elements. United States Patent, Number 5,542,043, 1996.

[8]

D. M. Cohen, S. R. Dalal, J. Parelius, and G. C. Patton. The combinatorial design approach to automatic test generation. IEEE Software, 13(5):82--88, October 1996.

Digital Library

[9]

M. B. Cohen, C. J. Colbourn, P. B. Gibbons, and W. B. Mugridge. Constructing test suites for interaction testing. Proc. Intl. Conf. on Software Engineering (ICSE 2003), pages 38--48, 2003.

Digital Library

[10]

M. B. Cohen, C. J. Colbourn, and A. C. H. Ling. Constructing strength three covering arrays with augmented annealing. Discrete Mathematics, to appear.

[11]

C. J. Colbourn. Combinatorial aspects of covering arrays. Le Matematiche (Catania), to appear.

[12]

C. J. Colbourn, M. B. Cohen, and R. C. Turban. A deterministic density algorithm for pairwise interaction coverage. Proc. of the IASTED Intl. Conference on Software Engineering, pages 242--252, February 2004.

[13]

C. J. Colbourn, Y. Chen, W.-T. Tsai. Progressive Ranking and Composition of Web Services Using Covering Arrays. Tenth IEEE International Workshop of Object-oriented Real-time Dependable Systems (WORDS2005), to appear.

Digital Library

[14]

S. R. Dalal, A. Jain, N. Karunanithi, J. M. Leaton, C. M. Lott, G. C. Patton, B. M. Horowitz. Model-based testing in practice. Proc. Intl. Conf. on Software Engineering (ICSE '99), pages 285--294, May 1999.

Digital Library

[15]

S. Dunietz, W. K. Ehrlich, B. D. Szablak, C. L. Mallows, and A. Iannino. Applying design of experiments to software testing. Proc. Intl. Conf. on Software Engineering (ICSE '97), pages 205--215, October 1997.

Digital Library

[16]

A. Dumitrescu. Efficient algorithms for generation of combinatorial covering suites. Proc. 14-th Annual Intl. Symp. Algorithms and Computation (ISAAC '03), Lecture Notes in Computer Science, pages 300--308, 2003.

[17]

S. Elbaum, D. Gable, and G. Rothermel. Understanding and measuring the sources of variation in the prioritization of regression test suites. In Proceedings of the 7th International Software Metrics Symposium, pages 169--179 Apr. 2001.

Digital Library

[18]

S. Elbaum, A. Malishevsky, and G. Rothermel. Using fault estimation to improve test case prioritization. Technical Report 99-60-13, Oregon State University, Corvallis, OR, Feb. 2000.

[19]

S. Elbaum, A. Malishevsky, and G. Rothermel. Incorporating varying test costs and fault severities into test case prioritization. In Proceedings of the 23rd International Conference on Software Engineering, pages 329--338, May 2001.

Digital Library

[20]

S. Elbaum, A. Malishevsky, and G. Rothermel. Prioritizing test cases for regression testing. In Proceedings of the International Symposium on Software Testing and Analysis, pages 102--112, Aug. 2001.

Digital Library

[21]

S. Elbaum, A. Malishevsky, and G. Rothermel. Test case prioritization: A family of empirical studies. IEEE Transactions on Software Engineering, 18(2):159--182, Feb. 2002.

Digital Library

[22]

A. Hartman. Software and hardware testing using combinatorial covering suites. Haifa Workshop on Interdisciplinary Applications of Graph Theory, Combinatorics, and Algorithms, June 2002.

[23]

A. Hartman and L. Raskin. Problems and algorithms for covering arrays. Discrete Math., 284:149--156, 2004.

Digital Library

[24]

N. Kobayashi, T. Tsuchiya, and T. Kikuno. A new method for constructing pairwise covering designs for software testing. Information Processing Letters, 81:85--91, 2002.

Digital Library

[25]

D. Kuhn and M. Reilly. An investigation of the applicability of design of experiments to software testing. Proc. 27th Annual NASA Goddard/IEEE Software Engineering Workshop, pages 91--95, October 2002.

Digital Library

[26]

D. R. Kuhn, D. R. Wallace, and A. M. Gallo. Software fault interactions and implications for software testing. IEEE Trans. Software Engineering, 30(6):418--421, October 2004.

Digital Library

[27]

R. Mandl. Orthogonal latin squares an application of experiment design to compiler testing. Communications of the ACM, 28(10):1054--1058, 1985.

Digital Library

[28]

National Institute of Standards and Technology. The Economic Impacts of Inadequate Infrastructure for Software Testing. U.S. Department of Commerce, May 2002.

[29]

K. Nurmela. Upper bounds for covering arrays by tabu search. Discrete Applied Math., 138(9):143--152, March 2004.

Digital Library

[30]

G. Rothermel, S. Elbaum, A. G. Malishevsky, P. Kallakuri, and X. Qiu. On test suite composition and cost-effective regression testing. ACM Transactions on Software Engineering and Methodology, 13(3):277--331, July 2004.

Digital Library

[31]

G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Test case prioritization: An empirical study. In Proceedings of the International Conference on Software Maintenance, pages 179--188, Sept. 1999.

Digital Library

[32]

K. Tai and L. Yu. A test generation strategy for pairwise testing. IEEE Transactions on Software Engineering, 28:109--111, 2002.

Digital Library

[33]

Y. Tung and W. Aldiwan. Automating test case generation for the new generation mission software system. IEEE Aerospace Conf., pages 431--37, 2000.

[34]

L. White and H. Almezen. Generating test cases for GUI responsibilities using complete interaction sequences. In Proc. of the Interactional Symposium on Software Reliability Engineering, pp.110--121, 2000.

Digital Library

[35]

A. W. Williams and R. L. Probert. A measure for component interaction test coverage. Proc. ACS/IEEE Intl. Conf. on Computer Systems and Applications, pages 301--311, October 2001.

Digital Library

[36]

W. E. Wong, J. R. Horgan, S. London, and H. Agrawal. A study of effective regression testing in practice. In Proceedings of the Eighth International Symposium on Software Reliability Engineering., pages 230--238, Nov. 1997.

Digital Library

[37]

C. Yilmaz, M. B. Cohen, and A. Porter. Covering arrays for efficient fault characterization in complex configuration spaces. Intl. Symp. on Software Testing and Analysis, pages 45--54, July 2004.

Digital Library

Cited By

Huang RSun WChen TTowey DChen JZong WZhou Y(2020)Abstract Test Case Prioritization Using Repeated Small-Strength Level-Combination CoverageIEEE Transactions on Reliability10.1109/TR.2019.290806869:1(349-372)Online publication date: Mar-2020
https://doi.org/10.1109/TR.2019.2908068
Kampel LSimos D(2019)A survey on the state of the art of complexity problems for covering arraysTheoretical Computer Science10.1016/j.tcs.2019.10.019Online publication date: Oct-2019
https://doi.org/10.1016/j.tcs.2019.10.019
Bures MAhmed B(2017)On the Effectiveness of Combinatorial Interaction Testing: A Case Study2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C)10.1109/QRS-C.2017.20(69-76)Online publication date: Jul-2017
https://doi.org/10.1109/QRS-C.2017.20
Show More Cited By

Index Terms

Test prioritization for pairwise interaction coverage
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

A framework of greedy methods for constructing interaction test suites
ICSE '05: Proceedings of the 27th international conference on Software engineering

Greedy algorithms for the construction of software interaction test suites are studied. A framework is developed to evaluate a large class of greedy methods that build suites one test at a time. Within this framework are many instantiations of greedy ...
Test suite prioritization by interaction coverage
DOSTA '07: Workshop on Domain specific approaches to software test automation: in conjunction with the 6th ESEC/FSE joint meeting

Event-driven software (EDS) is a widely used class of software that takes sequences of events as input, changes state, and outputs new event sequences. Managing the size of tests suites for EDS is difficult as the number of event combinations and ...
Test prioritization for pairwise interaction coverage

Interaction testing is widely used in screening for faults. In software testing, it provides a natural mechanism for testing systems to be deployed on a variety of hardware and software configurations. Several algorithms published in the literature are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

A-MOST '05: Proceedings of the 1st international workshop on Advances in model-based testing

May 2005

136 pages

ISBN:1595931155

DOI:10.1145/1083274

ACM SIGSOFT Software Engineering Notes Volume 30, Issue 4
July 2005
1514 pages
ISSN:0163-5948
DOI:10.1145/1082983
Issue’s Table of Contents

Copyright © 2005 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 May 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
869
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 20 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang RSun WChen TTowey DChen JZong WZhou Y(2020)Abstract Test Case Prioritization Using Repeated Small-Strength Level-Combination CoverageIEEE Transactions on Reliability10.1109/TR.2019.290806869:1(349-372)Online publication date: Mar-2020
https://doi.org/10.1109/TR.2019.2908068
Kampel LSimos D(2019)A survey on the state of the art of complexity problems for covering arraysTheoretical Computer Science10.1016/j.tcs.2019.10.019Online publication date: Oct-2019
https://doi.org/10.1016/j.tcs.2019.10.019
Bures MAhmed B(2017)On the Effectiveness of Combinatorial Interaction Testing: A Case Study2017 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C)10.1109/QRS-C.2017.20(69-76)Online publication date: Jul-2017
https://doi.org/10.1109/QRS-C.2017.20
Ahmed BZamli KAfzal WBures M(2017)Constrained Interaction Testing: A Systematic Literature StudyIEEE Access10.1109/ACCESS.2017.27715625(25706-25730)Online publication date: 2017
https://doi.org/10.1109/ACCESS.2017.2771562
Huang RZong WChen JTowey DZhou YChen D(2016)Prioritizing Interaction Test Suites Using Repeated Base Choice Coverage2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC)10.1109/COMPSAC.2016.167(174-184)Online publication date: Jun-2016
https://doi.org/10.1109/COMPSAC.2016.167
Huang RChen JTowey DChan ALu Y(2015)Aggregate-strength interaction test suite prioritizationJournal of Systems and Software10.1016/j.jss.2014.09.00299:C(36-51)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.jss.2014.09.002
Gao SLv JDu BColbourn CMa S(2015)Balancing Frequencies and Fault Detection in the In-Parameter-Order AlgorithmJournal of Computer Science and Technology10.1007/s11390-015-1574-630:5(957-968)Online publication date: 14-Sep-2015
https://doi.org/10.1007/s11390-015-1574-6
HUANG RTOWEY DCHEN JLU Y(2014)New Metrics for Prioritized Interaction Test SuitesIEICE Transactions on Information and Systems10.1587/transinf.E97.D.830E97.D:4(830-841)Online publication date: 2014
https://doi.org/10.1587/transinf.E97.D.830
Huang RChen JLi ZWang RLu YCho YShin SKim SHung CHong J(2014)Adaptive random prioritization for interaction test suitesProceedings of the 29th Annual ACM Symposium on Applied Computing10.1145/2554850.2554854(1058-1063)Online publication date: 24-Mar-2014
https://dl.acm.org/doi/10.1145/2554850.2554854
Wu HNie CKuo F(2014)Test Suite Prioritization by Switching CostProceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation Workshops10.1109/ICSTW.2014.15(133-142)Online publication date: 31-Mar-2014
https://dl.acm.org/doi/10.1109/ICSTW.2014.15
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents