More Web Proxy on the site http://driver.im/

research-article

Did we test our changes?: assessing alignment between tests and development in practice

Authors:

Sebastian Eder,

Benedikt Hauptmann,

Maximilian Junker,

Elmar Juergens,

Karl-Heinz PrommerAuthors Info & Claims

AST '13: Proceedings of the 8th International Workshop on Automation of Software Test

Pages 107 - 110

Published: 18 May 2013 Publication History

Abstract

Testing and development are increasingly performed by different organizations, often in different countries and time zones. Since their distance complicates communication, close alignment between development and testing becomes increasingly challenging. Unfortunately, poor alignment between the two threatens to decrease test effectiveness or increases costs.

In this paper, we propose a conceptually simple approach to assess test alignment by uncovering methods that were changed but never executed during testing. The paper's contribution is a large industrial case study that analyzes development changes, test service activity and field faults of an industrial business information system over 14 months. It demonstrates that the approach is suitable to produce meaningful data and supports test alignment in practice.

References

[1]

N. Nagappan and T. Ball, "Use of relative code churn measures to predict system defect density," in ICSE, 2005.

Digital Library

[2]

N. Nagappan, B. Murphy, and V. Basili, "The influence of organizational structure on software quality," in ICSE, 2008.

Digital Library

[3]

T. Graves, A. Karr, J. Marron, and H. Siy, "Predicting fault incidence using software change history," IEEE Trans. Softw. Eng., vol. 26, no. 7, 2000.

Digital Library

[4]

T. J. Ostrand, E. J. Weyuker, and R. M. Bell, "Where the bugs are," in ISSTA, 2004.

Digital Library

[5]

T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell, "A systematic literature review on fault prediction performance in software engineering," IEEE Trans. Softw. Eng., vol. 38, no. 6, 2012.

Digital Library

[6]

N. Nagappan, A. Zeller, T. Zimmermann, K. Herzig, and B. Murphy, "Change bursts as defect predictors," in ISSRE, 2010.

Digital Library

[7]

V. Channakeshava, V. K. Shanbhag, A. Panigrahi, R. Sisodia, and S. Lakshmanan, "Safe subset-regression test selection for managed code," in ISEC, 2008.

Digital Library

[8]

Y.-F. Chen, D. Rosenblum, and K.-P. Vo, "Testtube: a system for selective regression testing," in ICSE, 1994.

Digital Library

[9]

T. L. Graves, M. J. Harrold, J.-M. Kim, A. Porter, and G. Rothermel, "An empirical study of regression test selection techniques," in ICSE, 1998.

Digital Library

[10]

M. Harrold and A. Orso, "Retesting software during development and maintenance," in FoSM, 2008.

[11]

A. Srivastava and J. Thiagarajan, "Effectively prioritizing tests in development environment," in ISSTA, 2002.

Digital Library

[12]

H. Zhu, P. A. V. Hall, and J. H. R. May, "Software unit test coverage and adequacy," ACM Comput. Surv., vol. 29, no. 4, 1997.

Digital Library

[13]

Q. Yang, J. J. Li, and D. Weiss, "A survey of coverage based testing tools," in AST, 2006.

Digital Library

[14]

Y. Malaiya, M. Li, J. Bieman, and R. Karcich, "Software reliability growth with test coverage," IEEE Trans. Rel., vol. 51, no. 4, 2002.

[15]

G. Rothermel, R. Untch, C. Chu, and M. Harrold, "Prioritizing test cases for regression testing," IEEE Trans. Softw. Eng., vol. 27, no. 10, 2001.

Digital Library

[16]

IEEE, "IEEE Standard Glossary of Software Engineering Terminology," New York, USA, 1990.

[17]

S. Eder, M. Junker, E. Jurgens, B. Hauptmann, R. Vaas, and K. Prommer, "How much does unused code matter for maintenance?" in ICSE, 2012.

Digital Library

[18]

O. Traub, S. Schechter, and M. D. Smith, "Ephemeral instrumentation for lightweight program profiling," School of engineering and Applied Sciences, Harvard University, Tech. Rep., 2000.

[19]

E. Juergens, M. Feilkas, M. Herrmannsdoerfer, F. Deissenboeck, R. Vaas, and K. Prommer, "Feature profiling for evolving systems," in ICPC, 2011.

Digital Library

Cited By

Rott JNiedermayr RJuergens EPagano DTonelli RCounsell SDestefanis GMurgia ATempero E(2017)Ticket coverageProceedings of the 8th Workshop on Emerging Trends in Software Metrics10.5555/3106039.3106041(2-8)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3106039.3106041
Junker MBroy MHauptmann BBöhm WFemmer HEder SJuergens EJanßen RVaas RShukla RSen SShull FBishop J(2015)Principles and a process for successful industry cooperationProceedings of the Second International Workshop on Software Engineering Research and Industrial Practice10.5555/2821387.2821400(50-53)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2821387.2821400
Eder SFemmer HHauptmann BJunker MBjarnason EBorg MMorandini M(2015)Configuring latent semantic indexing for requirements tracingProceedings of the Second International Workshop on Requirements Engineering and Testing10.5555/2820704.2820713(27-33)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2820704.2820713

Did we test our changes?: assessing alignment between tests and development in practice

Recommendations

Excluding code from test coverage: practices, motivations, and impact
Abstract
Test coverage measures the percentage of code that is covered (and uncovered) by tests. In practice, not all code is equally important for coverage analysis, like code that will not be executed during tests. Some coverage tools provide support for ...
Test Polarity: Detecting Positive and Negative Tests
FSE 2024: Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering

Positive tests (aka, happy path tests) cover the expected behavior of the program, while negative tests (aka, unhappy path tests) check the unexpected behavior. Ideally, test suites should have both positive and negative tests to better protect against ...
A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

AST '13: Proceedings of the 8th International Workshop on Automation of Software Test

May 2013

155 pages

ISBN:9781467361613

Program Chairs:
Hong Zhu
Oxford Brookes University
,
Henry Muccini
University of L'Aquila, Italy
,
Zhenyu Chen
Nanjing University, Nanjing, China

Sponsors

Publisher

IEEE Press

Publication History

Published: 18 May 2013

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '13

Sponsor:

SIGSOFT

ICSE '13: 35th International Conference on Software Engineering

May 18 - 19, 2013

California, San Francisco

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
57
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rott JNiedermayr RJuergens EPagano DTonelli RCounsell SDestefanis GMurgia ATempero E(2017)Ticket coverageProceedings of the 8th Workshop on Emerging Trends in Software Metrics10.5555/3106039.3106041(2-8)Online publication date: 20-May-2017
https://dl.acm.org/doi/10.5555/3106039.3106041
Junker MBroy MHauptmann BBöhm WFemmer HEder SJuergens EJanßen RVaas RShukla RSen SShull FBishop J(2015)Principles and a process for successful industry cooperationProceedings of the Second International Workshop on Software Engineering Research and Industrial Practice10.5555/2821387.2821400(50-53)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2821387.2821400
Eder SFemmer HHauptmann BJunker MBjarnason EBorg MMorandini M(2015)Configuring latent semantic indexing for requirements tracingProceedings of the Second International Workshop on Requirements Engineering and Testing10.5555/2820704.2820713(27-33)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2820704.2820713

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents