short-paper

Cost-Effective Traceability Links for Architecture-Level Software Understanding: A Controlled Experiment

Authors:

Muhammad Atif Javed,

Srdjan Stevanetic,

Uwe ZdunAuthors Info & Claims

ASWEC ' 15 Vol. II: Proceedings of the ASWEC 2015 24th Australasian Software Engineering Conference

Pages 69 - 73

https://doi.org/10.1145/2811681.2811695

Published: 28 September 2015 Publication History

Get Access

Abstract

An important architectural challenge is to recover traceability links between the software architecture and artifacts produced in the other activities of the development process, such as requirements, detailed design, architectural knowledge, and implementation. This is challenging because, on the one hand, it is desirable to recover traceability links of a high quality and at the right quantity for aiding the software architect or developer, but, on the other hand, the costs and efforts spent for recovering should be as low as possible. The literature suggests manual, semi-automatic, and automatic recovery methods, each of which exhibits different impacts on costs as well as quantity and quality of the recovered links. To date, however, none of the published empirical studies have comparatively examined the automation alternatives of traceability link recovery. This paper reports on a controlled experiment that was conducted to investigate how well typical results produced by the three automation alternatives support human software developers in architecture-level understanding of the software system. The results provide statistical evidence that a focus on automated information retrieval (IR) based traceability recovery methods significantly reduces the quantity and quality of the elements retrieved by the software developers, whereas no significant differences between manual and semi-automatic traceability link recovery were found.

References

[1]

D. Cuddeback, A. Dekhtyar, and J. Hayes. Automated requirements traceability: The study of human analysts. In Proceedings of the 18th International Requirements Engineering Conference, RE '10, pages 231--240. IEEE.

Digital Library

Google Scholar

[2]

G. A. A. Cysneiros, F. Andrea, and Z. G. Spanoudakis. Traceability approach for i* and uml models. In Proceedings of the 2nd International Workshop on Software Engineering for Large-Scale Multi-Agent Systems (SELMAS'03), 2003.

Google Scholar

[3]

A. Dekhtyar, O. Dekhtyar, J. Holden, J. Hayes, D. Cuddeback, and W.-K. Kong. On human analyst performance in assisted requirements tracing: Statistical analysis. In Proceedings of the 19th International Requirements Engineering Conference, RE '11, pages 111--120. IEEE.

Digital Library

Google Scholar

[4]

A. Egyed, S. Biffl, M. Heindl, and P. Grünbacher. A value-based approach for understanding cost-benefit trade-offs during automated software traceability. In Proceedings of the 3rd International Workshop on Traceability in Emerging Forms of Software Engineering, TEFSE 2005, pages 2--7. ACM.

Digital Library

Google Scholar

[5]

M. A. Javed and U. Zdun. On the effects of traceability links in differently sized software systems. In Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, EASE 2015. ACM.

Digital Library

Google Scholar

[6]

M. A. Javed and U. Zdun. The supportive effect of traceability links in architecture-level software understanding: Two controlled experiments. In Proceedings of the 11th Working IEEE/IFIP Conference on Software Architecture, WICSA 2014, pages 215--224. IEEE.

Digital Library

Google Scholar

[7]

M. A. Javed and U. Zdun. The supportive effect of traceability links in change impact analysis for evolving architectures -- two controlled experiments. In 14th International Conference on Software Reuse, ICSR 2015. Springer link.

Google Scholar

[8]

B. A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, and J. Rosenberg. Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering, 28(8):721--734, Aug. 2002.

Digital Library

Google Scholar

[9]

S. Klock, M. Gethers, B. Dit, and D. Poshyvanyk. Traceclipse: An eclipse plug-in for traceability link recovery and management. In Proceedings of the 6th International Workshop on Traceability in Emerging Forms of Software Engineering, TEFSE '11, pages 24--30. ACM.

Digital Library

Google Scholar

[10]

W. H. Kruskal and W. A. Wallis. Use of Ranks in One-Criterion Variance Analysis. volume 47, pages 583--621. American Statistical Association, 1952.

Google Scholar

[11]

H. Mann and D. Whitney. On a test of whether one of two random variables is stochastically larger than the other. volume 18, pages 50--60. Institute of Mathematical Statistics, 1947.

Crossref

Google Scholar

[12]

R. Oliveto, M. Gethers, D. Poshyvanyk, and A. De Lucia. On the equivalence of information retrieval methods for automated traceability link recovery. In Proceedings of the 18th International Conference on Program Comprehension, ICPC '10, pages 68--71. IEEE.

Digital Library

Google Scholar

[13]

C. Wohlin. Experimentation in Software Engineering: An Introduction: An Introduction. The Kluwer International Series in Software Engineering. Kluwer Academic, 2000.

Digital Library

Google Scholar

[14]

X. Zou, R. Settimi, and J. Cleland-Huang. Improving automated requirements trace retrieval: A study of term-based enhancement methods. Empirical Softw. Engg., 15(2):119--146, Apr. 2010.

Digital Library

Google Scholar

Cited By

View all

Javed MUL Muram FZdun U(2018)On-Demand Automated Traceability Maintenance and EvolutionNew Opportunities for Software Reuse10.1007/978-3-319-90421-4_7(111-120)Online publication date: 17-Apr-2018
https://doi.org/10.1007/978-3-319-90421-4_7
Javed MStevanetic SZdun UPreschern CEloranta V(2016)Towards a pattern language for construction and maintenance of software architecture traceability linksProceedings of the 21st European Conference on Pattern Languages of Programs10.1145/3011784.3011810(1-20)Online publication date: 6-Jul-2016
https://dl.acm.org/doi/10.1145/3011784.3011810

Index Terms

Cost-Effective Traceability Links for Architecture-Level Software Understanding: A Controlled Experiment
1. General and reference
  1. Cross-computing tools and techniques
    1. Metrics
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Software architectures

Recommendations

The Supportive Effect of Traceability Links in Architecture-Level Software Understanding: Two Controlled Experiments
WICSA '14: Proceedings of the 2014 IEEE/IFIP Conference on Software Architecture

The advocates of architecture traceability approaches regularly cite advantages like easier understanding of architectural designs and support for software quality control and maintenance. However, the lack of published empirical data on the usefulness ...
Do developers benefit from requirements traceability when evolving and maintaining a software system?

Software traceability is a required component of many software development processes. Advocates of requirements traceability cite advantages like easier program comprehension and support for software maintenance (i.e., software change). However, despite ...
Dependencies, traceability and consistency in software architecture: towards a view-based perspective
ECSA '11: Proceedings of the 5th European Conference on Software Architecture: Companion Volume

Managing dependencies between software development artifacts helps achieve traceability and consistency in software-intensive systems. Traceability supports the alignment between different stakeholder concerns, development artifacts, and various ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ASWEC ' 15 Vol. II: Proceedings of the ASWEC 2015 24th Australasian Software Engineering Conference

September 2015

171 pages

ISBN:9781450337960

DOI:10.1145/2811681

Editors:
Fei-Ching (Diana) Kuo
Swinburne University of Technology, Australia
,
Stuart Marshall
Victoria University of Wellington, New Zealand
,
Haifeng Shen
Flinders University, Australia
,
Markus Stumptner
University of South Australia, Australia
,
M. Ali Babar
University of Adelaide, Australia

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Australian Comp Soc: Australian Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Funding Sources

Austrian Science Fund

Conference

ASWEC ' 15 Vol. II

ASWEC ' 15 Vol. II: ASWEC 2015 24th Australasian Software Engineering Conference

September 28 - October 1, 2015

SA, Adelaide, Australia

Acceptance Rates

ASWEC ' 15 Vol. II Paper Acceptance Rate 12 of 27 submissions, 44%;

Overall Acceptance Rate 12 of 27 submissions, 44%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
96
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Javed MUL Muram FZdun U(2018)On-Demand Automated Traceability Maintenance and EvolutionNew Opportunities for Software Reuse10.1007/978-3-319-90421-4_7(111-120)Online publication date: 17-Apr-2018
https://doi.org/10.1007/978-3-319-90421-4_7
Javed MStevanetic SZdun UPreschern CEloranta V(2016)Towards a pattern language for construction and maintenance of software architecture traceability linksProceedings of the 21st European Conference on Pattern Languages of Programs10.1145/3011784.3011810(1-20)Online publication date: 6-Jul-2016
https://dl.acm.org/doi/10.1145/3011784.3011810

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

The Supportive Effect of Traceability Links in Architecture-Level Software Understanding: Two Controlled Experiments

Do developers benefit from requirements traceability when evolving and maintaining a software system?

Dependencies, traceability and consistency in software architecture: towards a view-based perspective