Article

Talking tests: an empirical assessment of the role of fit acceptance tests in clarifying requirements

Authors:

Filippo Ricca,

Marco Torchiano,

Mariano Ceccato,

Paolo TonellaAuthors Info & Claims

IWPSE '07: Ninth international workshop on Principles of software evolution: in conjunction with the 6th ESEC/FSE joint meeting

Pages 51 - 58

https://doi.org/10.1145/1294948.1294962

Published: 03 September 2007 Publication History

Get Access

Abstract

The starting point for software evolution is usually a change request, expressing the new or updated requirements on the delivered system. The requirements specified in a change request document are often incomplete and inconsistent with the initial requirement document, as well as the implementation. Programmers working on the evolution of the software are often in trouble interpreting an under-specified change request document, resulting in code that does not meet the users' expectations and contains faults that can only be detected later through expensive testing activities.

In this paper, we investigate the role of acceptance tests to clarify the requirements used in software evolution iterations. In particular we focus on Fit tables, a way to express acceptance tests which simplifies their translation into executable test cases. We designed and ran an experiment to assess whether availability of Fit tables affects the level of understanding and the productivity in understanding the requirements. Results indicate that Fit tables significantly improve requirement understanding, but tend to involve additional effort.

References

[1]

J. Aarniala. Acceptance testing. In whitepaper. www.cs.helsinki.fi/u/jaarnial/jaarnial-testing.pdf, October 30 2006.

Google Scholar

[2]

R. Callan. Building Object-Oriented Systems: An Introduction from Concepts to Implementation in C++. WIT Press (UK); BkDisk edition, 1994.

Google Scholar

[3]

N. Juristo and A. Moreno. Basics of Software Engineering Experimentation. Kluwer Academic Publishers, Englewood Cliffs, NJ, 2001.

Digital Library

Google Scholar

[4]

G. Melnik, K. Read, and F. Maurer. Suitability of fit user acceptance tests for specifying functional requirements: Developer perspective. In Extreme programming and agile methods -- XP/Agile Universe 2004, pages 60--72, August 2004.

Google Scholar

[5]

B. Meyer. On formalism in specification. IEEE Software, January 1985.

Digital Library

Google Scholar

[6]

R. Mugridge and W. Cunningham. Fit for Developing Software: Framework for Integrated Tests. Prentice Hall, 2005.

Digital Library

Google Scholar

[7]

A. N. Oppenheim. Questionnaire Design, Interviewing and Attitude Measurement. Pinter, London, 1992.

Google Scholar

[8]

K. Read, G. Melnik, and F. Maurer. Examining usage patters of the fit acceptance testing framework. In Proc. 6th International Conference on eXtreme Programming and Agile Processes in Software Engineering (XP2005), pages Lecture Notes in Computer Science, Vol. 3556, Springer Verlag: 127--136 2005, June 18--23 2005.

Digital Library

Google Scholar

[9]

F. Ricca, M. D. Penta, M. Torchiano, P. Tonella, and M. Ceccato. The role of experience and ability in comprehension tasks supported by uml stereotypes. In Proceedings of the International Conference on Software Engineering (ICSE), pages 375--384. IEEE Computer Society, May 2007.

Digital Library

Google Scholar

[10]

C. Wohlin, P. Runeson, M. Höst, M. Ohlsson, B. Regnell, and A. Wesslén. Experimentation in Software Engineering -- An Introduction. Kluwer Academic Publishers, 2000.

Digital Library

Google Scholar

[11]

R. Young. Effective Requirements Practice. Addison-Wesley, Boston, MA, 2001.

Digital Library

Google Scholar

Cited By

View all

Hotomski SGlinz M(2018)A Qualitative Study on using GuideGen to Keep Requirements and Acceptance Tests Aligned2018 IEEE 26th International Requirements Engineering Conference (RE)10.1109/RE.2018.00-54(29-39)Online publication date: Aug-2018
https://doi.org/10.1109/RE.2018.00-54
Weiss JSchill ARichter IMandl P(2016)Literature Review of Empirical Research Studies within the Domain of Acceptance Testing2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2016.33(181-188)Online publication date: Aug-2016
https://doi.org/10.1109/SEAA.2016.33
Weiss JSchill A(2016)Specifying Executable or Nonexecutable Acceptance Test Cases for Event Processing Applications2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2016.17(235-239)Online publication date: Aug-2016
https://doi.org/10.1109/SEAA.2016.17
Show More Cited By

Index Terms

Talking tests: an empirical assessment of the role of fit acceptance tests in clarifying requirements
1. Software and its engineering
  1. Software creation and management
    1. Designing software
      1. Requirements analysis
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Using acceptance tests as a support for clarifying requirements: A series of experiments

One of the main reasons for the failure of many software projects is the late discovery of a mismatch between the customers' expectations and the pieces of functionality implemented in the delivered system. At the root of such a mismatch is often a set ...
Tests and Requirements, Requirements and Tests: A Möbius Strip

Writing acceptance tests early is a requirements-engineering technique that can save businesses time and money and help them better respond to change. An equivalence hypothesis states that concrete requirements blend with acceptance tests so that you ...
Are fit tables really talking?: a series of experiments to understand whether fit tables are useful during evolution tasks
ICSE '08: Proceedings of the 30th international conference on Software engineering

Test-driven software development tackles the problem of operationally defining the features to be implemented by means of test cases. This approach was recently ported to the early development phase, when requirements are gathered and clarified. Among ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IWPSE '07: Ninth international workshop on Principles of software evolution: in conjunction with the 6th ESEC/FSE joint meeting

September 2007

122 pages

ISBN:9781595937223

DOI:10.1145/1294948

Program Chairs:
Massimiliano Di Penta
RCOST --- Università degli Studi del Sannio, Italy
,
Michele Lanza
University of Lugano, Switzerland

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ESEC/FSE07

Sponsor:

ACM
SIGSOFT
CEPIS

ESEC/FSE07: Joint 11th European Software Engineering Conference 2007

September 3 - 4, 2007

Dubrovnik, Croatia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
298
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hotomski SGlinz M(2018)A Qualitative Study on using GuideGen to Keep Requirements and Acceptance Tests Aligned2018 IEEE 26th International Requirements Engineering Conference (RE)10.1109/RE.2018.00-54(29-39)Online publication date: Aug-2018
https://doi.org/10.1109/RE.2018.00-54
Weiss JSchill ARichter IMandl P(2016)Literature Review of Empirical Research Studies within the Domain of Acceptance Testing2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2016.33(181-188)Online publication date: Aug-2016
https://doi.org/10.1109/SEAA.2016.33
Weiss JSchill A(2016)Specifying Executable or Nonexecutable Acceptance Test Cases for Event Processing Applications2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)10.1109/SEAA.2016.17(235-239)Online publication date: Aug-2016
https://doi.org/10.1109/SEAA.2016.17
Weiss JMandl PSchill A(2015)Executable Test Specifications Show No Negative Effect on Finding and Correcting Test Faults in Event Processing Application DevelopmentProceedings of the 2015 41st Euromicro Conference on Software Engineering and Advanced Applications10.1109/SEAA.2015.16(18-26)Online publication date: 26-Aug-2015
https://dl.acm.org/doi/10.1109/SEAA.2015.16
Sieber SAmjath-Babu TMcIntosh BTscherning KMüller KHelming KPohle DFricke KVerweij PPacini CJansson TGomez y Paloma S(2013)Evaluating the characteristics of a non-standardised Model Requirements Analysis (MRA) for the development of policy impact assessment toolsEnvironmental Modelling & Software10.5555/2772070.277211649:C(53-63)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.5555/2772070.2772116
Belfo F(2012)Software Requirements Management through the Lenses of People, Organizational and Technological DimensionsInternational Journal of Web Portals10.4018/jwp.20120701044:3(47-61)Online publication date: 1-Jul-2012
https://doi.org/10.4018/jwp.2012070104
Belfo F(2012)People, Organizational and Technological Dimensions of Software Requirements SpecificationProcedia Technology10.1016/j.protcy.2012.09.0345(310-318)Online publication date: 2012
https://doi.org/10.1016/j.protcy.2012.09.034
Ricca FTorchiano MDi Penta MCeccato MTonella P(2009)Using acceptance tests as a support for clarifying requirementsInformation and Software Technology10.1016/j.infsof.2008.01.00751:2(270-283)Online publication date: 1-Feb-2009
https://dl.acm.org/doi/10.1016/j.infsof.2008.01.007
Ricca FDi Penta MTorchiano MTonella PCeccato MVisaggio CSchäfer WDwyer MGruhn V(2008)Are fit tables really talking?Proceedings of the 30th international conference on Software engineering10.1145/1368088.1368138(361-370)Online publication date: 15-May-2008
https://dl.acm.org/doi/10.1145/1368088.1368138
Ricca FDi Penta MTorchiano M(2008)Guidelines on the use of Fit tables in software maintenance tasks: Lessons learned from 8 experiments2008 IEEE International Conference on Software Maintenance10.1109/ICSM.2008.4658080(317-326)Online publication date: Sep-2008
https://doi.org/10.1109/ICSM.2008.4658080

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Using acceptance tests as a support for clarifying requirements: A series of experiments

Tests and Requirements, Requirements and Tests: A Möbius Strip

Are fit tables really talking?: a series of experiments to understand whether fit tables are useful during evolution tasks