Abstract
To support debugging, maintenance, verification and validation (V&V) and/or independent V&V (IV&V), it is necessary to understand the relationship between defect reports and their related artifacts. For example, one cannot correct a code-related defect report without being able to find the code that is affected. Information retrieval (IR) techniques have been used effectively to trace textual artifacts to each other. This has generally been applied to the problem of dynamically generating a trace between artifacts in the software document hierarchy after the fact (after development has proceeded to at least the next lifecycle phase). The same techniques can also be used to trace textual artifacts of the software engineering lifecycle to defect reports. We have applied the term frequency–inverse document frequency (TF-IDF) technique with relevance feedback, as implemented in our requirements tracing on-target (RETRO) tool, to the problem of tracing textual requirement elements to related textual defect reports. We have evaluated the technique using a dataset for a NASA scientific instrument. We found that recall of over 85% and precision of 69%, and recall of 70% and precision of 99% could be achieved, respectively, on two subsets of the dataset.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Dekhtyar A, Hayes JH, Menzies T (2004) Text is software too. In: Proceedings of the international workshop on mining of software Repositories (MSR) 2004, Edinborough, Scotland, pp. 22–27
Hayes JH, Dekhtyar A, Osbourne J (2003) Improving requirements tracing via information retrieval. In: Proceedings of the international conference on requirements engineering (RE), Monterey, California, pp. 151–161
Hayes JH, Dekhtyar A, Sundaram KS, Howard S (2004) Helping analysts trace requirements: an objective look. In: Proceedings of the international conference on requirements engineering (RE), Kyoto, Japan
Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison-Wesley, Reading
Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41(6):391–407
Hayes JH, Dekhtyar A, Sundaram, KS (2005) Measuring the effectiveness of retrieval techniques in requirements tracing. In submission
MDP Website, CM-1 Project, http://mdp.ivv.nasa.gov/mdp_glossary.html#CM1
Measuring Search Effectiveness, http://www.hsl.creighton.edu/hsl/Searching/Recall-Precision.html, last accessed 8 April 2005.
Antoniol G, Canfora G, Casazza G, De Lucia A, Merlo E (2002) Recovering traceability links between code and documentation. IEEE Trans Softw Eng 28(10):970–983
Marcus A, Maletic J (2003) Recovering documentation-to-source code traceability links using latent semantic indexing. In: Proceedings of the 25th international conference on software engineering 2003, Portland, Oregon, pp. 125–135
Sundaram S, Hayes JH, Dekhtyar A (2005) Baselines in requirements tracing. In: To appear in proceedings of the international workshop on predictor models in software engineering (PROMISE 2005), in conjunction with the international conference on softare engineering, St. Louis
Chillarege R, Kao W, Condit RG (1991) Defect type and its impact on the growth curve. In: Proceedings of the 13th international conference on Software engineering (ICSE 1991), Austin, Texas, pp. 246—255
Leszak M, Perry DE, Stoll D (2000) A case study in root cause defect analysis. In: Proceedings of the 22nd international conference on Software engineering (ICSE 2000), Limerick, Ireland, pp. 428—437
Hayes JH (2003) Building a requirement fault taxonomy: experiences from a NASA verification and validation research project. In: Proceedings of the IEEE international symposium on software reliability engineering (ISSRE), Denver, pp. 49–59
Kuhn DR (1999) Fault classes and error detection capability of specification-based testing. ACM Trans Softw Eng Methodol 8(4):411–424
von Mayrhauser A, Wang J, Ohlsson MC, Wohlin C (1999) deriving a fault architecture from defect history. In: Proceedings of the international symposium on software reliability engineering, ISSRE99, Boca Raton, Florida, pp. 295–303
Chillarege R, Bhandafi I, Chaar J, Halliday M, Moebus D, Ray B, Wong M (1992) Orthogonal defect classification a concept for in-process measurements. IEEE TSE 18(11):943–956
Lutz R (1993) analyzing software requirements errors in safety-critical, embedded systems. In: Proceedings of the 1st IEEE international symposium on requirements engineering, pp. 126–133
Munson JC, Nikora AP (2002) Toward a quantifiable definition of software faults. In: 13th international symposium on software reliability engineering, pp. 388–395
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yadla, S., Hayes, J. & Dekhtyar, A. Tracing requirements to defect reports: an application of information retrieval techniques. Innovations Syst Softw Eng 1, 116–124 (2005). https://doi.org/10.1007/s11334-005-0011-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11334-005-0011-3