More Web Proxy on the site http://driver.im/

research-article

Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise

Authors:

Dag I.K. SjobergAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 33, Issue 2

Pages 65 - 86

https://doi.org/10.1109/TSE.2007.17

Published: 01 February 2007 Publication History

Abstract

A total of 295 junior, intermediate, and senior professional Java consultants (99 individuals and 98 pairs) from 29 international consultancy companies in Norway, Sweden, and the UK were hired for one day to participate in a controlled experiment on pair programming. The subjects used professional Java tools to perform several change tasks on two alternative Java systems with different degrees of complexity. The results of this experiment do not support the hypotheses that pair programming in general reduces the time required to solve the tasks correctly or increases the proportion of correct solutions. On the other hand, there is a significant 84 percent increase in effort to perform the tasks correctly. However, on the more complex system, the pair programmers had a 48 percent increase in the proportion of correct solutions but no significant differences in the time taken to solve the tasks correctly. For the simpler system, there was a 20 percent decrease in time taken but no significant differences in correctness. However, the moderating effect of system complexity depends on the programmer expertise of the subjects. The observed benefits of pair programming in terms of correctness on the complex system apply mainly to juniors, whereas the reductions in duration to perform the tasks correctly on the simple system apply mainly to intermediates and seniors. It is possible that the benefits of pair programming will exceed the results obtained in this experiment for larger, more complex tasks and if the pair programmers have a chance to work together over a longer period of time.

References

[1]

E. Arisholm and D.I.K. Sjøberg, ”A Controlled Experiment with Professionals to Evaluate the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software,” Technical Report 2003-6, Simula Research Laboratory, http://www.simula.no/~erika, 10 Sept. 2003.

[2]

E. Arisholm and D.I.K. Sjøberg, ”Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software,” IEEE Trans. Software Eng., vol. 30, no. 8, pp. 521-534, Aug. 2004.

Digital Library

[3]

E. Arisholm, D.I.K. Sjøberg, G.J. Carelius, and Y. Lindsjørn, ”A Web-Based Support Environment for Software Engineering Experiments,” Nordic J. Computing, vol. 9, no. 4, pp. 231-247, 2002.

Digital Library

[4]

K. Beck, “Embrace Change with Extreme Programming,” Computer, vol. 32, no. 10, pp. 70-77, Oct. 1999.

Digital Library

[5]

K. Beck and C. Andres, Extreme Programming Explained: Embrace Change, second ed. Addison Wesley, 2004.

Digital Library

[6]

C.F. Bond and L.J. Titus, ”Social Facilitation: A Meta-Analysis of 241 Studies,” Psychological Bull., vol. 94, no. 2, pp. 265-292, 1983.

[7]

R. Brown, Group Processes, second ed. Blackwell, 2000.

[8]

J.-M. Burkhardt, F. Detienne, and S. Wiedenbeck, “Object-Oriented Program Comprehension: Effect of Expertise, Task and Phase,” Empirical Software Eng., vol. 7, no. 2, pp. 115-156, 2002.

Digital Library

[9]

L. Cao and P. Xu, ”Activity Patterns of Pair Programming,” Proc. 38th Hawaii Int'l Conf. System Sciences, 2005.

Digital Library

[10]

A. Cockburn and L. Williams, ”The Costs and Benefits of Pair Programming,” Extreme Programming Examined, G. Succi and M.Marchesi, eds., Addison Wesley, 2001.

Digital Library

[11]

J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed. Lawrence Erlbaum Assoc., 1988.

[12]

J. Cohen, ”A Power Primer,” Psychological Bull., vol. 112, no. 1, pp.155-159, 1992.

[13]

L.L. Constantine, Constantine on Peopleware. Prentice Hall, 1995.

[14]

T.D. Cook and D.T. Campbell, Quasi-Experimentation—Design & Analysis Issues for Field Settings. Houghton Mifflin, 1979.

[15]

J.O. Coplien, ”A Generative Development-Process Pattern Language,” Pattern Languages of Program Design, J.O. Coplien and D.C.Schmidt, eds., pp. 183-237, Addison-Wesley, 1995.

Digital Library

[16]

T. Dybå, V.B. Kampenes, and D.I.K. Sjøberg, ”A Systematic Review of Statistical Power in Software Engineering Experiments,” Information and Software Technology, vol. 48, no. 8, pp. 745-755, 2006.

[17]

N.V. Flor and E.L. Hutchins, ”Analyzing Distributed Cognition in Software Teams: A Case Study of Team Programming During Perfective Software Maintenance,” Proc. Fourth Workshop Empirical Studies of Programmers, pp. 36-64, 1991.

[18]

D.R. Forsyth, Group Dynamics, third ed., Wadsworth, 1999.

[19]

H. Gallis, E. Arisholm, and T. Dybå, ”An Initial Framework for Research on Pair Programming,” Proc. 2003 ACM-IEEE Int'l Symp. Empirical Software Eng. (ISESE '03), pp. 132-142, 2003.

Digital Library

[20]

S. Heiberg, U. Puus, P. Salumaa, and A. Seeba, “Pair-Programming Effect on Developers Productivity,” Proc. Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp.215-224, 2003.

Digital Library

[21]

S. Holm, ”A Simple Sequentially Rejective Multiple Test Procedure,” Scandinavian J. Statistics, vol. 6, pp. 65-70, 1979.

[22]

T. Hærem, ”Task Complexity and Expertise as Determinants of Task Perceptions and Performance,” PhD dissertation, Norwegian School of Management BI, 2002.

[23]

S.J. Karau and K.D. Williams, ”Social Loafing: A Meta-Analytic Review and Theoretical Integration,” J. Personality and Social Psychology, vol. 65, no. 4, pp. 681-706, 1993.

[24]

B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K.E. Emam, and J. Rosenberg, ”Preliminary Guidelines for Empirical Research in Software Engineering,” IEEE Trans. Software Eng., vol. 28, no. 8, pp. 721-734, Aug. 2002.

Digital Library

[25]

K.M. Lui and K.C.C. Chan, “When Does a Pair Outperform Two Individuals?” Proc. Int'l Conf. Extreme Programming and Agile Processes in Software Eng., pp. 225-233, 2003.

Digital Library

[26]

K.M. Lui and K.C.C. Chan, ”A Cognitive Model for Solo Programming and Pair Programming,” Proc. Int'l Conf. Cognitive Informatics (ICCI '04), 2004.

Digital Library

[27]

C. McDowell, L. Werner, H. Bullock, and J. Fernald, ”The Effects of Pair-Programming on Performance in an Introductory Programming Course,” Proc. 33rd SIGCSE Technical Symp. Computer Science Education, pp. 38-42, 2002.

Digital Library

[28]

R.H. Myers, D.C. Montgomery, and G.G. Vining, Generalized Linear Models: With Applications in Engineering and the Sciences, first ed. Wiley-Interscience, 2001.

[29]

M.M. Müller, ”Are Reviews an Alternative to Pair Programming?” Empirical Software Eng., vol. 9, no. 4, pp. 335-351, 2004.

Digital Library

[30]

M.M. Müller, “Two Controlled Experiments Concerning the Comparison of Pair Programming to Peer Review,” J. Systems and Software, vol. 78, no. 2, pp. 166-179, 2005.

Digital Library

[31]

J. Nawrocki and A. Wojciechowski, ”Experimental Evaluation of Pair Programming,” Proc. European Software Control and Metrics Conf. (ESCOM), 2001.

[32]

J.T. Nosek, ”The Case for Collaborative Programming,” Comm. ACM, vol. 41, no. 3, pp. 105-108, 1998.

Digital Library

[33]

F. Padberg and M.M. Müller, ”Analyzing the Cost and Benefit of Pair Programming,” Proc. Ninth Int'l Software Metrics Symp., pp.166-177, 2003.

Digital Library

[34]

W.R. Shadish, T.D. Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton-Mifflin, 2002.

[35]

S.D. Sheetz, ”Identifying the Difficulties of Object-Oriented Development,” J. Systems and Software, vol. 64, no. 1, pp. 23-36, 2002.

Digital Library

[36]

D.I.K. Sjøberg, B. Anda, E. Arisholm, T. Dybå, M. Jørgensen, A. Karahasanovic, and M. Vokác, “Challenges and Recommendations when Increasing the Realism of Controlled Software Engineering Experiments,” Empirical Methods and Studies in Software Engineering: Experiences from ESERNET, R. Conradi and A.I.Wang, eds., pp. 24-38, Springer-Verlag, 2003.

[37]

E. Soloway, R. Lampert, S. Letowski, D. Littman, and J. Pinto, “Designing Documentation to Compensate for Delocalized Plans,” Comm. ACM, vol. 31, no. 11, pp. 1259-1267, 1988.

Digital Library

[38]

B.W. Tuckman, ”Development Sequence in Small Groups,” Psychological Bull., vol. 63, no. 6, pp. 384-399, 1965.

[39]

M. Vokác, W. Tichy, D.I.K. Sjøberg, E. Arisholm, and M. Aldrin, ”A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns— AReplication in a Real Programming Environment,” Empirical Software Eng., vol. 9, no. 3, 2004.

Digital Library

[40]

G.M. Weinberg, The Psychology of Computer Programming. Van Nostrand Reinhold, 1971.

Digital Library

[41]

L. Williams and R.L. Upchurch, ”In Support of Student Pair-Programming,” Proc. 32nd SIGCSE Technical Symp. Computer Science Education, pp. 327-331, 2001.

Digital Library

[42]

L. Williams and R. Kessler, Pair Programming Illuminated. Addison-Wesley, 2002.

Digital Library

[43]

L. Williams, A. Shukla, and A.I. Antón, ”An Initial Exploration of the Relationship between Pair Programming and Brooks' Law,” Proc. Agile Development Conf. (ADC '04), pp. 11-20, 2004.

Digital Library

[44]

L. Williams, R.R. Kessler, W. Cunningham, and R. Jeffries, ”Strengthening the Case for Pair Programming,” IEEE Software, vol. 17, no. 4, pp. 19-25, 2000.

Digital Library

[45]

L. Williams, E. Wiebe, K. Yang, M. Ferzli, and C. Miller, ”In Support of Pair Programming in the Introductory Computer Science Course,” Computer Science Education, vol. 12, no. 3, pp. 197-212, 2002.

[46]

L.A. Williams, ”The Collaborative Software Process,” PhD dissertation, Univ. of Utah, 2000.

Digital Library

[47]

R.J. Wirfs-Brock, ”Characterizing Your Application's Control Style,” Report on Object Analysis and Design, vol. 1, no. 3, 1994.

[48]

R.J. Wirfs-Brock and B. Wilkerson, ”Object-Oriented Design: A Responsibility Driven Approach,” SIGPLAN Notices, vol. 24, no. 10, pp. 71-75, 1989.

Digital Library

[49]

C.F.J. Wu and M. Hamada, Experiments: Planning, Analysis, and Parameter Design Optimization, first ed. Wiley-Interscience, 2000.

[50]

R.B. Zajonc, “Social Facilitation,” Science, no. 149, pp. 269-274, 1965.

Cited By

Cui JZhang RLi RZhou FSong YGehringer EStephenson BStone JBattestilli LRebelsky SShoop L(2024)How Pre-class Programming Experience Influences Students' Contribution to Their Team Project: A Statistical StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630870(255-261)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630870
Lemos OSilveira FFerrari FSilva TGuerra EGarcia A(2024)Unraveling the code: an in-depth empirical study on the impact of development practices in auxiliary functions implementationSoftware Quality Journal10.1007/s11219-024-09682-432:3(1137-1174)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09682-4
Kitchenham BMadeyski L(2024)Recommendations for analysing and meta-analysing small sample size software engineering experimentsEmpirical Software Engineering10.1007/s10664-024-10504-129:6Online publication date: 17-Aug-2024
https://dl.acm.org/doi/10.1007/s10664-024-10504-1
Show More Cited By

Index Terms

Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise

Recommendations

Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software

A fundamental question in object-oriented design is how to design maintainable software. According to expert opinion, a delegated control style, typically a result of responsibility-driven design, represents object-oriented design at its best, whereas a ...
The Impact of Static and Dynamic Pairs on Pair Programming
SERE-C '14: Proceedings of the 2014 IEEE Eighth International Conference on Software Security and Reliability-Companion

Pair programming is a programming technique in which pair of people doing programming on a single machine. The programming pair consists of a driver and a navigator or observer. The driver addresses the operational issues; i.e., responsible for ...
Evaluating Gender Significance within a Pair Programming Context
HICSS '13: Proceedings of the 2013 46th Hawaii International Conference on System Sciences

This study reports findings of the gender differences within a pair programming context. A large pool of university computer programming course undergraduate and graduate students was paired into three distinct pair categories: male-male, male-female, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 33, Issue 2

February 2007

77 pages

ISSN:0098-5589

Issue’s Table of Contents

Copyright © 2007.

Publisher

IEEE Press

Publication History

Published: 01 February 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cui JZhang RLi RZhou FSong YGehringer EStephenson BStone JBattestilli LRebelsky SShoop L(2024)How Pre-class Programming Experience Influences Students' Contribution to Their Team Project: A Statistical StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630870(255-261)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630870
Lemos OSilveira FFerrari FSilva TGuerra EGarcia A(2024)Unraveling the code: an in-depth empirical study on the impact of development practices in auxiliary functions implementationSoftware Quality Journal10.1007/s11219-024-09682-432:3(1137-1174)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11219-024-09682-4
Kitchenham BMadeyski L(2024)Recommendations for analysing and meta-analysing small sample size software engineering experimentsEmpirical Software Engineering10.1007/s10664-024-10504-129:6Online publication date: 17-Aug-2024
https://dl.acm.org/doi/10.1007/s10664-024-10504-1
Idowu SOsman OStrüber DBerger T(2024)Machine learning experiment management tools: a mixed-methods empirical studyEmpirical Software Engineering10.1007/s10664-024-10444-w29:4Online publication date: 29-May-2024
https://dl.acm.org/doi/10.1007/s10664-024-10444-w
Aljehane SSharif BMaletic J(2023)Studying Developer Eye Movements to Measure Cognitive Workload and Visual Effort for Expertise AssessmentProceedings of the ACM on Human-Computer Interaction10.1145/35911357:ETRA(1-18)Online publication date: 18-May-2023
https://dl.acm.org/doi/10.1145/3591135
Kitchenham BMadeyski LBudgen D(2023)SEGRESS: Software Engineering Guidelines for REporting Secondary StudiesIEEE Transactions on Software Engineering10.1109/TSE.2022.317409249:3(1273-1298)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/TSE.2022.3174092
Fagerlund JVesisenaho MHäkkinen P(2022)Fourth grade students’ computational thinking in pair programming with ScratchInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2022.10051133:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.ijcci.2022.100511
Dominic JTubre BKunkel DRodeghero P(2022)The human experience of comprehending source code in virtual realityEmpirical Software Engineering10.1007/s10664-022-10196-527:7Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10664-022-10196-5
Kuttal SOng BKwasny KRobe PKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Trade-offs for Substituting a Human with an Agent in a Pair Programming Context: The Good, the Bad, and the UglyProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445659(1-20)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445659
Zieris FPrechelt LJuristo N(2021)Two elements of pair programming skillProceedings of the 43rd International Conference on Software Engineering: New Ideas and Emerging Results10.1109/ICSE-NIER52604.2021.00019(51-55)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1109/ICSE-NIER52604.2021.00019
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents