research-article

Evaluating Software Design Processes by Analyzing Change Data Over Time

Authors:

L. J. Chmura,

T. J. Wicinski,

A. F. NorcioAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 16, Issue 7

Pages 729 - 740

https://doi.org/10.1109/32.56099

Published: 01 July 1990 Publication History

Publisher Site

Abstract

An analysis is presented of early design and code change data from the software cost reduction (SCR) project, a well-reported effort conducted at the US Naval Research Laboratory from 1978 to 1988. The analyses are mostly time-based studies of the change data and relationships between the data and SCR personnel activity data. Some analyses of the change data show patterns consistent with a major goal of the SCR project: the design and development of easy-to-change software. Specifically, most changes took a day or less to uncover and resolve; the majority of changes updated at most one module. Moreover, these percentages remained fairly stable. No positive relationship appeared between error-correction effort and the number of days that an error remained in the SCR design documentation. Other analyses suggest that consistency may have been temporary. For example, the analyses suggest a stepwise growth in average change effort, and an increasing percentage of changes resulted in module interface updates. Certain specific ratios between SCR change data and personnel activity data may be possible indicators of design incompleteness.

References

[1]

{1} V. R. Basili and B. T. Perricone, "Software errors and complexity: An empirical investigation," Commun. ACM, vol. 27, pp. 42-52. 1984.

Digital Library

Google Scholar

[2]

{2} V. R. Basili and D. M. Weiss, "'A methodology for collecting valid software engineering data," IEEE Trans. Software Eng., vol. SE-10. no. 6, pp. 728-738, 1984.

Digital Library

Google Scholar

[3]

{3} L. A. Belady and M. M. Lehman, "A model of large program development," IBM Syst. J., vol. 3, pp. 225-252, 1976.

Digital Library

Google Scholar

[4]

{4} B. W. Boehm, Software Engineering Economics. Englewood Cliffs, NJ: Prentice-Hall, 1981.

Crossref

Google Scholar

[5]

{5} K. H. Britton and D. L. Parnas, A-7E Module Guide, Naval Res. Lab., Washington, DC, Rep. 4702, 1981.

Google Scholar

[6]

{6} L. J. Chmura, "Proposed new design and code change report form (CRF) for the software cost reduction (SCR) project" Naval Res. Lab., Washington, DC, Tech. Memo. 7590-34:LC, 1983.

Google Scholar

[7]

{7} L. J. Chmura and A. F. Norcio, "Accuracy of software activity data The software cost reduction project," Naval Res. Lab., Washington, DC, Rep. 8780, 1983.

Google Scholar

[8]

{8} L. J, Chmura and D. M. Weiss, "The A-7E software requirements document: Three years of change data," Naval Res. Lab., Washington, DC, Memo. Rep. 4938, 1982.

Google Scholar

[9]

{9} V. Church, D. Card, F. McGarry, and V. Basili, "Guide to data collection," NASA/Goddard SEL, Greenbelt, MD, Tech. Rep. SEL- 81-001, 1981.

Google Scholar

[10]

{10} P. C. Clements, "Software cost reduction through disciplined design," Naval Res. Lab. 1984 Rev., pp. 79-87, 1985.

Google Scholar

[11]

{11} P. C. Clements, R. A. Parker, D. L. Parnas, and J. Shore, "A standard organization for specifying abstract interfaces," Naval Res. Lab., Washington, DC, Rcp. 8815, 1984.

Google Scholar

[12]

{12} R. Day, "A history of software maintenance for a complex U.S. Army battlefield automated system," in Proc. Conf. Software Maintenance, 1985, pp. 181-187.

Crossref

Google Scholar

[13]

{13} E. W. Dijkstra, "Cooperating sequential processes," in Programming Languages, F. Genuys, Ed. New York: Academic. 1968, pp. 43-112.

Google Scholar

[14]

{14} W. J. Dixon and F. J. Massey, Jr., Introduction to Statistical Analysis . New York: McGraw-Hill, 1969.

Google Scholar

[15]

{15} A. Endres, "An analysis of errors and their causes in system programs," IEEE Trans. Software Eng., vol. SE-1, no. 2, pp. 140-149, 1975.

Digital Library

Google Scholar

[16]

{16} R. B. Grady, "Measuring and managing software maintenance," IEEE Software, pp. 35-45 Sept. 1987.

Digital Library

Google Scholar

[17]

{17} K. L. Heninger, J. W. Kallander, J. E. Shore, D. L, Parnas, and Staff, "Software requirements for the A-7E aircraft," Naval Res. Lab., Washington, DC, Memo. Rep. 3876, 1978.

Google Scholar

[18]

{18} B. P. Lientz, E. B. Swanson, and G. E. Tompkins, "Characteristics of application software maintenance," Commun. ACM, vol. 21, no. 6, pp. 466-471, 1978.

Digital Library

Google Scholar

[19]

{19} B. Meyer, "On formalism in specifications," IEEE Software, vol. 2, no. 1, pp. 6-27, 1985.

Digital Library

Google Scholar

[20]

{20} A. F. Norcio and L. J. Chmura, "Design activity in developing modules for complex software," in Empirical Studies of Programmers, E. Soloway and S. Iyengar, Eds. Norwood, NJ: Ablex, 1986.

Crossref

Google Scholar

[21]

{21} A. Parker, K. L. Heninger, D. Parnas, and J. Shore, "Abstract interface specification for the device interface module," Naval Res. Lab., Washington, DC, Memo. Rep. 4385, 1980.

Google Scholar

[22]

{22} D. L. Parnas, "On the criteria to be used in decomposing systems into modules," Commun. ACM, vol. 15, pp. 1053-1058, 1972.

Digital Library

Google Scholar

[23]

{23} D. L. Parnas, "Use of abstract interfaces in the development of software for embedded systems," Naval Res. Lab., Washington, DC, Rep. 8047, 1977.

Google Scholar

[24]

{24} D. L. Parnas and P. C. Clements, "A rational design process" How and why to fake it," IEEE Trans. Software Eng., vol. SE-12, no. 2, pp. 251-257, 1986.

Crossref

Google Scholar

[25]

{25} D. L. Parnas, P. C. Clements, and D, M. Weiss, "The modular structure of complex systems," IEEE Trans. Software Eng., vol. SE- 11, no. 3, pp. 259-266, 1985.

Digital Library

Google Scholar

[26]

{26} M. L. Shooman and M. I. Bolsky, "Types, distribution, and test and correction times for programming errors," in Proc. Int. Conf. Reliable Software, pp. 347-357.

Crossref

Google Scholar

[27]

{27} E. B. Swanson, "The dimensions of maintenance," in Proc. Second Int. Conf. Software Engineering, 1976, pp. 492-497.

Crossref

Google Scholar

[28]

{28} D. M. Weiss, "A comparison of errors in different software-development environments" Naval Res. Lab., Washington, DC, NRL Rep. 8598, 1982.

Google Scholar

[29]

{29} D. M. Weiss and V. R. Basili, "Evaluating software development by analysis of changes Some data from the Software Engineering Laboratory," IEEE Trans. Software Eng., vol. SE 11, no. 2, pp. 157- 168, 1985.

Digital Library

Google Scholar

[30]

{30} R. Z. Xu, "An empirical investigation" Software errors and their influence upon development of WPADT," in Proc. COMPSAC 85, 1985, pp. 4-8.

Google Scholar

Cited By

View all

Cook JWolf A(1999)Software process validationACM Transactions on Software Engineering and Methodology10.1145/304399.3044018:2(147-176)Online publication date: 1-Apr-1999
https://dl.acm.org/doi/10.1145/304399.304401
Cook JVotta LWolf A(1998)Cost-Effective Analysis of In-Place Software ProcessesIEEE Transactions on Software Engineering10.1109/32.70770024:8(650-663)Online publication date: 1-Aug-1998
https://dl.acm.org/doi/10.1109/32.707700
Munson JKhoshgoftaar T(1992)The Detection of Fault-Prone ProgramsIEEE Transactions on Software Engineering10.1109/32.13577518:5(423-433)Online publication date: 1-May-1992
https://dl.acm.org/doi/10.1109/32.135775

Index Terms

Evaluating Software Design Processes by Analyzing Change Data Over Time

Recommendations

On the concurrent behaviour of SCR specifications
COMPSAC '97: Proceedings of the 21st International Computer Software and Applications Conference

The SCR method models a system using a set of variables and state machines whose behaviour is described by tabular notations. This paper proposes an interleaving semantics for SCR specifications in terms of timed transition systems. The semantics is ...
Impact analysis in the software change process: a year 2000 perspective
ICSM '96: Proceedings of the 1996 International Conference on Software Maintenance

Software change impact analysis has gained considerable attention with recent challenges of the Year 2000 Date situation. As the software community recognizes the growing need to identify consequences of these changes, impact analysis is making its way ...
Using EVOWAVE to Analyze Software Evolution
ICEIS 2015: Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 2

Software evolution produces large amounts of data which software engineers need to understand for their

daily activities. The use of software visualization constitutes a promising approach to help them comprehend

multiple aspects of the evolving ...

Reviews

Reviewer: David Noel Card

The authors describe an approach to analyzing software measurement data by looking at trends within a single project rather than attempting to make comparisons across projects. This approach effectively counters the usual project manager refrain of “my project is unique—you don't want to collect data from it because you can't compare it to anything else.” The reader interested in the practical application of measurement to software engineering will find food for thought in this paper. The paper presents an analysis of software design effort and change data from the A-7 project at the Naval Research Laboratory. Counts of changes, errors, design hours, and review hours were combined to form several different ratios. Values of the ratios were plotted by module over time. The behaviors of two ratios were found to be related to the establishment of design baselines. The authors collected a lot of data as part of their study, and they present 28 figures and six tables worth of it in this paper. Most of the trends identified by the authors are suggestive rather than statistically conclusive. Still, the authors present more data than they discuss. At times that interferes with getting across what they actually learned from the study.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 16, Issue 7

July 1990

99 pages

ISSN:0098-5589

Editor:
Victor R. Basili
Univ. of Maryland, College Park

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 July 1990

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cook JWolf A(1999)Software process validationACM Transactions on Software Engineering and Methodology10.1145/304399.3044018:2(147-176)Online publication date: 1-Apr-1999
https://dl.acm.org/doi/10.1145/304399.304401
Cook JVotta LWolf A(1998)Cost-Effective Analysis of In-Place Software ProcessesIEEE Transactions on Software Engineering10.1109/32.70770024:8(650-663)Online publication date: 1-Aug-1998
https://dl.acm.org/doi/10.1109/32.707700
Munson JKhoshgoftaar T(1992)The Detection of Fault-Prone ProgramsIEEE Transactions on Software Engineering10.1109/32.13577518:5(423-433)Online publication date: 1-May-1992
https://dl.acm.org/doi/10.1109/32.135775

Abstract

References

Cited By

Index Terms

Recommendations

On the concurrent behaviour of SCR specifications

Impact analysis in the software change process: a year 2000 perspective

Using EVOWAVE to Analyze Software Evolution

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations