More Web Proxy on the site http://driver.im/

Article

PBR vs. checklist: a replication in the n-fold inspection context

Authors:

Jeffrey CarverAuthors Info & Claims

ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering

Pages 95 - 104

https://doi.org/10.1145/1159733.1159750

Published: 21 September 2006 Publication History

Abstract

Inspection is considered a powerful method to check software documents for defects. Many published work shows that inspections in requirements specification phase are particularly effective and efficient. Perspective-Based Reading (PBR) is one of the systematic techniques to support defect detection in requirements documents. In this paper we describe an experiment to validate the effectiveness of PBR in a meeting-based N-fold inspection. Our goals were: (1) re-test the hypothesis of the original experiment that PBR helps to increase individual and team defect detection effectiveness compared to an checklist approach; (2) investigate the different impact of PBR and checklist on the effectiveness of N-fold team meeting; and (3) investigate some interesting characteristics of PBR (e.g. the relationship between background experiences and performance of the subjects). The results of the study showed that PBR was significantly more effective than checklist (supporting the original study). We also found that the team meeting is much more important for checklist teams, based on the number of meeting gains and the number of false defects eliminated. Finally, we found that teams using the PBR techniques have less overlap in their defect detection than those using checklist. The ultimate goal is to provide best practices (guidance) for applying PBR in software inspection and also some advice for PBR (or software inspections) process improvement.

References

[1]

Ballman, K., and Votta, L.G. Organizational congestion in large-scale software development. In Proceeding of the Third International Conference on the Software Process. IEEE CS Press, Los Alamitos, CA, 1994, 123--134.

[2]

Basili, V. R., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sørumgård, S., and Zelkowitz, M. V. The Empirical Investigation of Perspective-Based Reading. Empirical Software Engineering: An International Journal, 1, 2 (1996), 133--164.

[3]

Basili, V. R., Selby, R., and Hutchens, D. Experimentation in Software Engineering. IEEE Transactions on Software Engineering, 12, 7 (Jul. 1986), 733--743.

Digital Library

[4]

Ciolkowski, M., Differding, C., Laitenberger, O., and Munch, J. Empirical Investigation of Perspective-Based Reading: A Replicated Experiment, Technical Report ISERN-97-13, Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany, 1997.

[5]

Fagan, M. Design and code inspections to reduce errors in program development. IBM System Journal, 15, 3 (1976), 182--211.

Digital Library

[6]

Freedman, D., and Weinberg, G. Handbook of Walkthroughs Inspections, and Technical Reviews. Dorset House Publishing, 1990.

Digital Library

[7]

Gilb, T. and Graham, D. Software Inspections. Addison-Wesley, London, England, 1993.

Digital Library

[8]

Humphrey, W. S. A Discipline for Software Engineering, Reading. Addison-Wesley, Mass., 1995.

Digital Library

[9]

Johnson, P., & Tjahjono, D. Assessing software review meetings: A controlled experiment study using CSRS. In Proceeding of the 19th International Conference on Software Engineering (ICSE'97). ACM Press, Los Angles, CA, 1997, 118--27.

Digital Library

[10]

Kamsties, E., and Lott, C. M. An Empirical Evaluation of Three Defect Detection Techniques. Proceedings of the Fifth European Software Engineering Conference, (Sitges, Barcelona, Spain, September 6-10, 1995). Springer, 1996, 362--383.

Digital Library

[11]

Knight, J. An Improved Inspection Technique. Communications of the ACM, 36, 11(Nov. 1993), 51--61.

Digital Library

[12]

Laitenberger, O., and DeBaud, J. An Encompassing Life-Cycle Centric Survey of Software Inspection. Technical Report ISERN-98-32, Fraunhofer Institute for Experimental Software Engineering, Kaiserslautern, Germany, 1998.

[13]

Lanubile, F. and Visaggio, G. Evaluating Defect Detection Techniques for Software Requirements Inspections, Technical Report ISERN-00-08, International Software Engineering Research Network.

[14]

Martin, J. and Tsai, .W. N-Fold Inspection: A Requirements Analysis Technique. Communications of the ACM, 33, 2(Feb. 1990), 223--232.

Digital Library

[15]

McCarthy, P., Porter, A., Sih, H., and Votta, L. An Experiment to Assess Cost Benefits of Inspection Meetings and their Alternatives: A Pilot Study. Proceedings of the 1996 IEEE International Software Metrics Symposium (Berlin, Germany, March 25-26, 1996). IEEE Computer Society Press, 1996, 100--111.

Digital Library

[16]

Perpich, J. M., Perry, D. E., and Porter, A. A. Anywhere, anytime, code inspections: Using the Web to remove inspection bottlenecks in large-scale software development. Proceeding of the 19th International Conference on Software Engineering (ICSE'97). ACM Press, Los Angles, CA, 1997, 14--21.

Digital Library

[17]

Porter, A., and Johnson, P. Assessing software review meeting: Results of a comparative analysis of two experimental studies. IEEE Transaction on Software Engineering, 23, 3 (1997), 129--145.

Digital Library

[18]

Porter, A., Votta, L. G., and Basili, V. R. Comparing Detection Methods for Software Requirements Inspections: A Replication Using Professional Subjects. Empirical Software Engineering, 3(1998), 355--379.

Digital Library

[19]

Porter, A., Votta, L. G. Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment.IEEE Transactions Software Eng., 21, 6 (Jun. 1995), 563--575.

Digital Library

[20]

Russel, G.W. (1991). Experience with inspection in ultralargescale developments. IEEE Software, 8, 1 (1991), 25--31.

Digital Library

[21]

Sapsomboon, B. Shared Defect Detection : The Shared Defect Detection: The Effects of Annotations in Asynchronous Software Inspection. Ph.D. Thesis, University of Pittsburg, Pittsburg, PA, 2000.

Digital Library

[22]

Schneider, G. M., Martin, J., and Tsai, W.T. An experimental study of fault detection in user requirements documents. ACM Transaction on Software Engineering, 1, 2, 188--204.

Digital Library

[23]

Shull, F. Developing Techniques for Using Software Documents: A Series of Empirical Studies. Ph.D. Thesis, University of Maryland, College Park, MD, 1998.

Digital Library

[24]

Shull, F., Carver, J., and Travassos, G. An Empirical Methodology for Introducing Software Processes. In proceedings of the Joint 8th European Software Engineering Conference (ESEC) and 9th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-9) (Vienna, Austria, September 10-14, 2001). ACM Press, 2004, 288--296.

Digital Library

[25]

Shull, F., Mendonça, M., Basili, V., Carver, J., Maldonado, J., Fabbri, S., Travassos, G., and Ferreira, M. Knowledge-Sharing Issues in Experimental Software Engineering. Empirical Software Engineering: An International Journal, 9, 1(Mar. 2004), 111--137.

Digital Library

[26]

Shull, F., Rus, I., and Basili, V. How Perspective-Based Reading Can improve Requirements Inspection. IEEE Software, 33, 7 (Jul. 2000), 73--79.

Digital Library

[27]

Stein, M., Riedl, J., Harner, S.J., & Mashayekhi, V. (1997). A case study of distributed, asynchronous software inspection. Proceeding of the 19th International Conference on Software Engineering (ICSE'97). ACM Press, Los Angeles, CA, 1997, 107--117.

Digital Library

[28]

Thompsom, C., & Riedl, J. (1995). Collaborative asynchronous inspection of software using Lotus Notes. Technical Report 95-047, Computer Science Department, University of Minnesota.

[29]

Tripp, L., Struck, W., and Pflung, B. The application of multiple team inspections on a safety-critical software standard. Proceeding of 4th Software Engineering Standards Application Workshop. IEEE CS Press, Los Alamitos, CA, 1991, 106--111

[30]

Votta, L. G. Does every inspection need a meeting? Proceedings of ACM SIGSOFT '93 Symposium on Foundations of Software Engineering (FSE'93) (Los Angeles, California, December 7-10, 1993).ACM Press, 1993, 107--114.

Digital Library

[31]

Wheeler, D. A., Brykczynski, B, and Meeson, R. N. Software Inspection: An Industry Best Practice. IEEE CS Press, Los Alamitos, CA, 1996.

Digital Library

[32]

Zhang, Z., Basili, V., and Shneiderman, B. Perspective-Based Usability Inspection: An Empirical Validation of Efficacy. Empirical Software Engineering - An International Journal, 4, 1(1999), 43--70.

Digital Library

Cited By

Daun MBrings J(2023)Investigating Factors Influencing Students’ Assessment of Conceptual ModelsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593960(470-474)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593960
Daun MBrings J(2023)Aggregating N-fold Requirements Inspection ResultsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593465(339-347)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593465
Li BSmidts C(2023)A Zone-based Model for Analysis of Dependent Failures in Requirements InspectionIEEE Transactions on Software Engineering10.1109/TSE.2023.3266157(1-18)Online publication date: 2023
https://doi.org/10.1109/TSE.2023.3266157
Show More Cited By

Index Terms

PBR vs. checklist: a replication in the n-fold inspection context
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management

Recommendations

Checklist Based Reading's Influence on a Developer's Understanding
ASWEC '08: Proceedings of the 19th Australian Conference on Software Engineering

This paper addresses the influence the Checklist Based Reading inspection technique has on a developer's ability to modify inspected code. Traditionally, inspections have been used to detect defects within the development life cycle. This research ...
Checklist-based Inspection of SMarty Variability Models
ICEIS 2015: Proceedings of the 17th International Conference on Enterprise Information Systems - Volume 2

Software inspection is a particular type of software review applied to all life-cycle artifacts and follows a

rigorous and well-defined defect detection process. Existing literature defines several inspection techniques

for different domains. However, ...
A Statistical Approach to the Inspection Checklist Formal Synthesis and Improvement

This paper proposes a statistical approach to the inspection checklist formal synthesis and improvement. The approach is based on a defect casual analysis and defect modeling. The defect model is developed using IBM's Orthogonal Defect Classification. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering

September 2006

388 pages

ISBN:1595932186

DOI:10.1145/1159733

General Chair:
Guilherme Horta Travassos
COPPE/UFRJ, Brazil
,
Program Chairs:
José Carlos Maldonado
ICMC-USP, Brazil
,
Claes Wohlin
Blekinge Institute of Technology, Sweden

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ISESE06

Sponsor:

ISESE06: ACM-IEEE International Symposium on Empirical Software Engineering 2006

September 21 - 22, 2006

Rio de Janeiro, Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
620
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Daun MBrings J(2023)Investigating Factors Influencing Students’ Assessment of Conceptual ModelsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593960(470-474)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593960
Daun MBrings J(2023)Aggregating N-fold Requirements Inspection ResultsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593465(339-347)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593465
Li BSmidts C(2023)A Zone-based Model for Analysis of Dependent Failures in Requirements InspectionIEEE Transactions on Software Engineering10.1109/TSE.2023.3266157(1-18)Online publication date: 2023
https://doi.org/10.1109/TSE.2023.3266157
Daun MBrings JWeyer T(2023)Model inspections in the engineering of collaborative cyber‐physical systems with instance‐level review diagramsJournal of Software: Evolution and Process10.1002/smr.239235:5Online publication date: 25-Apr-2023
https://dl.acm.org/doi/10.1002/smr.2392
Li BDiao XGao WSmidts C(2021)A requirements inspection method based on scenarios generated by model mutation and the experimental validationEmpirical Software Engineering10.1007/s10664-021-10001-926:5Online publication date: 3-Aug-2021
https://doi.org/10.1007/s10664-021-10001-9
Daun MBrings JKrajinski LWeyer TSutton SArmbrust OHebig R(2019)On the benefits of using dedicated models in validation processes for behavioral specificationsProceedings of the International Conference on Software and System Processes10.1109/ICSSP.2019.00016(44-53)Online publication date: 25-May-2019
https://dl.acm.org/doi/10.1109/ICSSP.2019.00016
Rodrigues LPrietch SMota MSerique BPrates RCandello H(2018)Analysis, Redesign and Validation of Accessibility Resources Applied to an Official Electronic Journal for the Promotion of Equal Access to Public ActsProceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems10.1145/3274192.3274200(1-10)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1145/3274192.3274200
Rodrigues AValentim NFeitosa EMota MSerique BPrates RCandello H(2018)A Set of Privacy Inspection Techniques for Online Social NetworksProceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems10.1145/3274192.3274195(1-11)Online publication date: 22-Oct-2018
https://dl.acm.org/doi/10.1145/3274192.3274195
Daun MSalmon AWeyer TPohl KLv JZhang HBabar M(2015)The impact of students' skills and experiences on empirical resultsProceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering10.1145/2745802.2745829(1-6)Online publication date: 27-Apr-2015
https://dl.acm.org/doi/10.1145/2745802.2745829
Alshazly AElfatatry AAbougabal M(2014)Detecting defects in software requirements specificationAlexandria Engineering Journal10.1016/j.aej.2014.06.00153:3(513-527)Online publication date: Sep-2014
https://doi.org/10.1016/j.aej.2014.06.001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents