research-article

How Effective Developers Investigate Source Code: An Exploratory Study

Authors:

Martin P. Robillard,

Wesley Coelho,

Gail C. MurphyAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 30, Issue 12

Pages 889 - 903

https://doi.org/10.1109/TSE.2004.101

Published: 01 December 2004 Publication History

Publisher Site

Abstract

Prior to performing a software change task, developers must discover and understand the subset of the system relevant to the task. Since the behavior exhibited by individual developers when investigating a software system is influenced by intuition, experience, and skill, there is often significant variability in developer effectiveness. To understand the factors that contribute to effective program investigation behavior, we conducted a study of five developers performing a change task on a medium-size open source system. We isolated the factors related to effective program investigation behavior by performing a detailed qualitative analysis of the program investigation behavior of successful and unsuccessful developers. We report on these factors as a set of detailed observations, such as evidence of the phenomenon of inattention blindness by developers skimming source code. In general, our results support the intuitive notion that a methodical and structured approach to program investigation is the most effective.

References

[1]

V.R. Basili R.W. Selby and D.H. Hutchens, “Experimentation in Software Engineering,” IEEE Trans. Software Eng., vol. 12, no. 7, pp. 733-743, July 1986.

Digital Library

Google Scholar

[2]

B.W. Boehm, “Software Engineering,” IEEE Trans. Computers, vol. 12,no. 25, pp. 1226-1242, Dec. 1976.

Digital Library

Google Scholar

[3]

L. Bratthall and M. Jørgensen, “Can You Trust a Single Data Source Exploratory Software Engineering Case Study?” Empirical Software Eng., vol. 7, no. 1, pp. 9-26, Mar. 2002.

Digital Library

Google Scholar

[4]

Y.-F. Chen M.Y. Nishimoto and C.V. Ramamoorthy, “The C Information Abstraction System,” IEEE Trans. Software Eng., vol. 16,no. 3, pp. 325-334, Mar. 1990.

Digital Library

Google Scholar

[5]

C.L. Corritore and S. Wiedenbeck, “Mental Representation of Expert Procedural and Object-Oriented Programmers in a Software Maintenance Task,” Int'l J. Human-Computer Studies, vol. 50, no. 1, pp. 61-83, Jan. 1999.

Digital Library

Google Scholar

[6]

C.L. Corritore and S. Wiedenbeck, “Direction and Scope of Comprehension-Related Activities by Procedural and Object-Oriented Programmers: An Empirical Study,” Proc. Eighth Int'l Workshop Program Comprehension, pp. 139-148, June 2000.

Digital Library

Google Scholar

[7]

C.L. Corritore and S. Wiedenbeck, “An Exploratory Study of Program Comprehension Strategies of Procedural and Object-Oriented Programmers,” Int'l J. Human-Computer Studies, vol. 54, no. 1, pp. 1-23, Jan. 2001.

Digital Library

Google Scholar

[8]

B. Curtis, “Substantiating Programmer Variability,” Proc. IEEE, vol. 69, no. 7, pp. 846, July 1981.

Crossref

Google Scholar

[9]

T. DeMarco and T. Lister, “Programmer Performance and the Effects of the Workplace,” Proc. Eighth Int'l Conf. Software Eng., pp.nbsp268-272, 1985.

Digital Library

Google Scholar

[10]

S.G. Eick T.L. Graves A.F. Karr J.S. Marron and A. Mockus, “Does Code Decay? Assessing the Evidence from Change Management Data,” IEEE Trans. Software Eng., vol. 27, no. 1, pp.nbsp1-12, 2001.

Digital Library

Google Scholar

[11]

A. Engebretson and S. Wiedenbeck, “Novice Comprehension of Program Using Task-Specific and Nontask-Specific Constructs,” Proc. IEEE 2002 Symp. Human Centric Computing Languages and Environments, pp. 11-18, Sept. 2002.

Digital Library

Google Scholar

[12]

A. Goldberg, Smalltalk-80: The Interactive Programming Environment. Addison-Wesley, 1984.

Digital Library

Google Scholar

[13]

B.M. Lange and T.G. Moher, “Some Strategies of Reuse in an Object-Oriented Programming Environment,” Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 69-73, 1989.

Digital Library

Google Scholar

[14]

M.M. Lehman and L.A. Belady, “Program Evolution: Processes of Software Change,” APIC Studies in Data Processing, vol. 27, 1985.

Digital Library

Google Scholar

[15]

M. Lejter S. Meyers and S.P. Reiss, “Support for Maintaining Object-Oriented Programs,” IEEE Trans. Software Eng., vol. 18,no. 12, pp. 1045-1052, Dec. 1992.

Digital Library

Google Scholar

[16]

S. Letovsky and E. Soloway, “Delocalized Plans and Program Comprehension,” IEEE Software, vol. 3, no. 3, pp. 41-49, May 1986.

Digital Library

Google Scholar

[17]

A. Mack and I. Rock, Inattentional Blindness. MIT Press, 1998.

Crossref

Google Scholar

[18]

R. Mosemann and S. Wiedenbeck, “Navigation and Comprehension of Programs by Novice Programmers,” Proc. Ninth Int'l Workshop Program Comprehension, pp. 79-88, May 2001.

Digital Library

Google Scholar

[19]

Object Technology International, Inc., “Eclipse Platform Technical Overview,” white paper, July 2001.

Google Scholar

[20]

P.D. O'Brien D.C. Halbert and M.F. Kilian, “The Trellis Programming Environment,” Proc. Conf. Object-Oriented Programming, Systems, and Applications, pp. 91-102, Oct. 1987.

Digital Library

Google Scholar

[21]

D.L. Parnas, “Sofware Aging,” Proc. 16th Int'l Conf. Software Eng., pp. 279-287, May 1994.

Digital Library

Google Scholar

[22]

S.L. Pfleeger, “Experimental Design and Analysis in Software Engineering-Part 3: Types of Experimental Design,” Software Eng. Notes, vol. 20, no. 2, pp. 14-16, Apr. 1995.

Digital Library

Google Scholar

[23]

D.F. Redmiles, “Reducing the Variability of Programmers' Performance Through Explained Examples,” Proc. Conf. Human Factors in Computing Systems, pp. 67-73, 1993.

Digital Library

Google Scholar

[24]

H. Sackman W.J. Erikson and E.E. Grant, “Exploratory Experimental Studies Comparing Online and Offline Programming Performance,” Comm. ACM, vol. 11, no. 1, pp. 3-11, 1968.

Digital Library

Google Scholar

[25]

M. Sanella, The Interlisp-D Reference Manual. Xerox Corporation, Palo Alto, Calif., 1983.

Google Scholar

[26]

C.B. Seaman, “Qualitative Methods in Empirical Studies of Software Engineering,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 557-572, July/Aug. 1999.

Digital Library

Google Scholar

[27]

J. Singer T. Lethbridge N. Vinson and N. Anquetil, “An Examination of Software Engineering Work Practices,” Proc. 1997 Conf. Centre for Advanced Studies on Collaborative Research, pp. 209-223, 1997.

Digital Library

Google Scholar

[28]

E. Soloway J. Pinto S. Letovsky D. Littman and R. Lampert, “Designing Documentation to Compensate for Delocalized Plans,” Comm. ACM, vol. 31, no. 11, pp. 1259-1267, Nov. 1988.

Digital Library

Google Scholar

[29]

B. Teasley L.M. Leventhal K. Instone and D.S. Rohlman, “Longitudinal Studies of the Relation of Programmer Expertise and Role-Expressiveness to Program Comprehension,” Proc. NATO Advanced Research Workshop User-Centred Requirements for Software Eng. Environments, NATO Advanced Science Institutes Series-Computer and Systems Science, vol. 123, pp. 143-163, Sept. 1991.

Google Scholar

[30]

W. Teitelman and L. Masinter, “The Interlisp Programming Environment,” Computer, vol. 14, no. 4, pp. 25-33, Apr. 1981.

Digital Library

Google Scholar

[31]

A.M. Vans A. von Mayrhauser and G. Somlo, “Program Understanding Behavior during Corrective Maintenance of Large-Scale Software,” Int'l J. Human-Computer Studies, vol. 51, no. 1, pp. 31-70, July 1999.

Digital Library

Google Scholar

[32]

A. von Mayrhauser and A.M. Vans, “Identification of Dynamic Comprehension Processes during Large Scale Maintenance,” IEEE Trans. Software Eng., vol. 22, no. 6, pp. 424-437, 1996.

Digital Library

Google Scholar

[33]

A. von Mayrhauser A.M. Vans and A.E. Howe, “Program Understanding Behaviour during Enhancement of Large-Scale Software,” J. Software Maintenance: Research and Practice, vol. 9,no. 5, pp. 299-327, Sept./Oct. 1997.

Digital Library

Google Scholar

[34]

S. Wiedenbeck V. Fix and J. Scholtz, “Characteristics of the Mental Representations of Novice and Expert Programmers: An Empirical Study,” Int'l J. Man-Machine Studies, vol. 39, pp. 793-812, 1993.

Digital Library

Google Scholar

[35]

R.K. Yin, “Case Study Research: Design and Methods,” Applied Social Research Methods Series, vol. 5, second ed. 1989.

Google Scholar

Cited By

View all

Cheng GZou DXie HWang F(2024)Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programmingComputers & Education10.1016/j.compedu.2023.104948208:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104948
Etaiwi LSager PGuéhéneuc YHamel S(2024)Consensus task interaction trace recommender to guide developers’ software navigationEmpirical Software Engineering10.1007/s10664-024-10528-729:6Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10664-024-10528-7
Goswami LSenges AEstier TCherubini M(2023)Supporting Co-Regulation and Motivation in Learning Programming in Online ClassroomsProceedings of the ACM on Human-Computer Interaction10.1145/36100897:CSCW2(1-29)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610089
Show More Cited By

Index Terms

How Effective Developers Investigate Source Code: An Exploratory Study
1. Software and its engineering
  1. Software creation and management
    1. Collaboration in software development
      1. Programming teams
    2. Software post-development issues
      1. Software reverse engineering
  2. Software notations and tools
    1. Software configuration management and version control systems

Recommendations

An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks

Much of software developers' time is spent understanding unfamiliar code. To better understand how developers gain this understanding and how software development environments might be involved, a study was performed in which developers were given an ...
What Do We Know about Agile Software Development?

From 1,996 studies available in the agile software development literature, the authors identified 36 research studies of acceptable rigor, credibility, and relevance to include in a systematic review of empirical evidence for agile development's ...
Impact of test-driven development on productivity, code and tests: A controlled experiment

Context: Test-driven development is an approach to software development, where automated tests are written before production code in highly iterative cycles. Test-driven development attracts attention as well as followers in professional environment; ...

Reviews

Reviewer: Ponmurugarajan Thiyagarajan

This work is an attempt to investigate the link between program investigation behavior and the success of a software modification task. The paper details an exploratory study conducted with five different developers performing a three-hour software modification task on a 65,000 lines of code (kLOC) text editor system written in Java. The authors critically analyze the results of the study, based on the behavior of successful and unsuccessful developers. Various hypotheses are derived based on various observations from the study. The authors did a good job of simulating a practical development environment for their study. Sufficient proofs and extra descriptions are provided throughout the paper to protect the integrity of the study, which was greatly appreciated. There are, however, a few points to criticize: first, today's software engineering practice is highly advanced and mature, with best practices and lessons learned, and, thus, I feel that many of the hypotheses provided by the authors as an effect of their study sound familiar. Second, I think the short development time (three hours) given to the developers is comparatively low for a modification task of 65 kLOC in an object-oriented development environment. This short time frame, I feel, must have had a significant effect on unsuccessful developers' behavior, and the results those developers produced might not be their best. Third, statistical justification for the selection of certain parameters, such as the number of developers (five), and the development time (three hours), has not been provided. Overall, the study was well conducted and the paper was nicely written.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 30, Issue 12

December 2004

256 pages

ISSN:0098-5589

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 December 2004

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

119
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cheng GZou DXie HWang F(2024)Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programmingComputers & Education10.1016/j.compedu.2023.104948208:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104948
Etaiwi LSager PGuéhéneuc YHamel S(2024)Consensus task interaction trace recommender to guide developers’ software navigationEmpirical Software Engineering10.1007/s10664-024-10528-729:6Online publication date: 2-Sep-2024
https://dl.acm.org/doi/10.1007/s10664-024-10528-7
Goswami LSenges AEstier TCherubini M(2023)Supporting Co-Regulation and Motivation in Learning Programming in Online ClassroomsProceedings of the ACM on Human-Computer Interaction10.1145/36100897:CSCW2(1-29)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610089
Ramasamy DSarasua CBacchelli ABernstein A(2023)Workflow analysis of data science code in public GitHub repositoriesEmpirical Software Engineering10.1007/s10664-022-10229-z28:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10664-022-10229-z
Rein PBeckmann TMattis THirschfeld R(2022)Toward Understanding Task Complexity in Maintenance-Based Studies of Programming ToolsCompanion Proceedings of the 6th International Conference on the Art, Science, and Engineering of Programming10.1145/3532512.3535223(38-45)Online publication date: 21-Mar-2022
https://dl.acm.org/doi/10.1145/3532512.3535223
Arab MLaToza TLiang JKo A(2022)An Exploratory Study of Sharing Strategic Programming KnowledgeProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502070(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502070
Razzaq AVentresque AKoschke RDe Lucia ABuckley J(2022)The Effect of Feature Characteristics on the Performance of Feature Location TechniquesIEEE Transactions on Software Engineering10.1109/TSE.2021.304973548:6(2066-2085)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1109/TSE.2021.3049735
Sharafi ZBertram IFlanagan MWeimer W(2022)Eyes on Code: A Study on Developers’ Code Navigation StrategiesIEEE Transactions on Software Engineering10.1109/TSE.2020.303206448:5(1692-1704)Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1109/TSE.2020.3032064
Horváth FBeszédes ÁVancsics BBalogh GVidács LGyimóthy T(2022)Using contextual knowledge in interactive fault localizationEmpirical Software Engineering10.1007/s10664-022-10190-x27:6Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s10664-022-10190-x
Bradley NFritz THolmes R(2022)Sources of software development task frictionEmpirical Software Engineering10.1007/s10664-022-10187-627:7Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10664-022-10187-6
Show More Cited By

Abstract

References

Cited By

Index Terms

Recommendations

An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks

What Do We Know about Agile Software Development?

Impact of test-driven development on productivity, code and tests: A controlled experiment

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations