research-article

An eye-tracking study on the role of scan time in finding source code defects

Authors:

Bonita Sharif,

Michael Falcone,

Jonathan I. MaleticAuthors Info & Claims

ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

Pages 381 - 384

https://doi.org/10.1145/2168556.2168642

Published: 28 March 2012 Publication History

Get Access

Abstract

An eye-tracking study is presented that investigates how individuals find defects in source code. This work partially replicates a previous eye-tracking study by Uwano et al. [2006]. In the Uwano study, eye movements are used to characterize the performance of individuals in reviewing source code. Their analysis showed that subjects who did not spend enough time initially scanning the code tend to take more time finding defects. The study here follows a similar setup with added eye-tracking measures and analyses on effectiveness and efficiency of finding defects with respect to eye gaze. The subject pool is larger and is comprised of a varied skill level. Results indicate that scanning significantly correlates with defect detection time as well as visual effort on relevant defect lines. Results of the study are compared and contrasted to the Uwano study.

References

[1]

Abdelnabi, Z., Cantone, G., Ciolkowski, M., and Rombach, D. 2004. Comparing Code Reading Techniques Applied to Object-Oriented Software Frameworks with regard to Effectiveness and Defect Detection Rate. in Proc. of Intl. Symposium on Empirical Software Engineering, pp. 239--248

Digital Library

Google Scholar

[2]

Bednarik, R. and Tukiainen, M. 2006. An Eye-tracking Methodology for Characterizing Program Comprehension Processes. in Proceedings of Symposium on Eye tracking research & Applications (ETRA), San Diego, pp. 125--132.

Digital Library

Google Scholar

[3]

Bednarik, R. and Tukiainen, M. 2008. Temporal Eye-tracking Data: Evolution of Debugging Strategies with Multiple Representations. in Proceedings of Symposium on Eye Tracking Research & Applications (ETRA), Savannah, pp. 99--102.

Digital Library

Google Scholar

[4]

Brooks, R. 1983. Towards a Theory of the Comprehension of Computer Programs. International Journal of Man-Machine Studies, vol. 18, no. 6, pp. 543--554.

Crossref

Google Scholar

[5]

Ciolkowski, M., Laitenberger, O. R., D., Shull, F., and Perry, D. 2002. Software Inspections, Reviews & Walkthroughs. in Proceedings of International Conference on Software Engineering (ICSE), pp. 641--642.

Digital Library

Google Scholar

[6]

Crosby, M. E. and Stelovsky, J. 1990. How do we read algorithms? A case study. IEEE Computer, v. 23, n. 1, pp. 24--35.

Digital Library

Google Scholar

[7]

Hundhausen, C., Agrawal, A., Fairbrother, D., and Trevisan, M. 2009. Integrating Pedagogical Code Reviews into a CS 1 Course: An Empirical Study. in Proc. of SIGCSE, Chattanooga, Tennessee, pp. 291--295.

Digital Library

Google Scholar

[8]

Pietinen, S., Bednarik, R., Glotova, T., Tenhunen, V., and Tukiainen, M. 2008. A Method to Study Visual Attention Aspects of Collaboration: Eye-tracking Pair Programmers Simultaneously. in Proceedings of Symposium on Eye Tracking Research & Applications, Savannah, Georgia, pp. 39--42.

Digital Library

Google Scholar

[9]

Sharif, B. and Maletic, J. I. 2010. An Eye tracking Study on CamelCase and Under_score Identifier Styles. in Proceedings of 18th IEEE Intl. Conf. on Program Comprehension, Braga, Jun 30-Jul 2, pp. 196--205.

Digital Library

Google Scholar

[10]

Uwano, H., Monden, A., and Matsumoto, K.-i. 2008. DRESREM 2: An Analysis System for Multi-document Software Review Using Reviewers' Eye Movements. in Proc. of Intl. Conf. on Software Engineering Advances, Malta, pp. 177--183

Digital Library

Google Scholar

[11]

Uwano, H., Nakamura, M., Monden, A., and Matsumoto, K. 2006. Analyzing individual performance of source code review using reviewers' eye movement. in Proc. of Symp. on Eye tracking research & applications, San Diego, pp.133--140

Digital Library

Google Scholar

Cited By

View all

Dempsey JTsiola ABosch NChristianson KStites M(2025)Eye-movement indices of reading while debugging Python source codeJournal of Cognitive Psychology10.1080/20445911.2024.2447117(1-19)Online publication date: 2-Jan-2025
https://doi.org/10.1080/20445911.2024.2447117
Zeng JLiu WZhou YLi XCheng WRuan Q(2024)A New Visualization Framework for Multimodal Programming Attention in Debugging TestProceedings of the ACM Turing Award Celebration Conference - China 202410.1145/3674399.3674422(35-41)Online publication date: 5-Jul-2024
https://dl.acm.org/doi/10.1145/3674399.3674422
Hauser FGrabinger LEzer TMottok JGruber H(2024)Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image PerceptionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655093(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655093
Show More Cited By

Index Terms

An eye-tracking study on the role of scan time in finding source code defects
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Process validation
        Walkthroughs

Recommendations

An eye-tracking study assessing the comprehension of c++ and Python source code
ETRA '14: Proceedings of the Symposium on Eye Tracking Research and Applications

A study to assess the effect of programming language on student comprehension of source code is presented, comparing the languages of C++ and Python in two task categories: overview and find bug tasks. Eye gazes are tracked while thirty-eight students ...
An Eye Tracking Study on camelCase and under_score Identifier Styles
ICPC '10: Proceedings of the 2010 IEEE 18th International Conference on Program Comprehension

An empirical study to determine if identifier-naming conventions (i.e., camelCase and under_score) affect code comprehension is presented. An eye tracker is used to capture quantitative data from human subjects during an experiment. The intent of this ...
Can the E-Z Reader Model Predict Eye Movements Over Code? Towards a Model of Eye Movements Over Source Code
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and Applications

Studies of eye movements during source code reading have supported the idea that reading source code differs fundamentally from reading natural text. The paper analyzed an existing data set of natural language and source code eye movement data using the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

March 2012

420 pages

ISBN:9781450312219

DOI:10.1145/2168556

Conference Chairs:
Carlos H. Morimoto
University of São Paulo, Brazil
,
Howell Istance
Montfort University, UK
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Jeffrey B. Mulligan
NASA
,
Pernilla Qvarfordt
FX Palo Alto

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 March 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ETRA '12

Sponsor:

ETRA '12: Eye Tracking Research and Applications

March 28 - 30, 2012

California, Santa Barbara

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

101
Total Citations
View Citations
888
Total Downloads

Downloads (Last 12 months)52
Downloads (Last 6 weeks)3

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Dempsey JTsiola ABosch NChristianson KStites M(2025)Eye-movement indices of reading while debugging Python source codeJournal of Cognitive Psychology10.1080/20445911.2024.2447117(1-19)Online publication date: 2-Jan-2025
https://doi.org/10.1080/20445911.2024.2447117
Zeng JLiu WZhou YLi XCheng WRuan Q(2024)A New Visualization Framework for Multimodal Programming Attention in Debugging TestProceedings of the ACM Turing Award Celebration Conference - China 202410.1145/3674399.3674422(35-41)Online publication date: 5-Jul-2024
https://dl.acm.org/doi/10.1145/3674399.3674422
Hauser FGrabinger LEzer TMottok JGruber H(2024)Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image PerceptionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655093(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655093
Yabesi SAmini MRistic JSharafi ZBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)Exploring the Effects of Urgency and Reputation in Code Review: An Eye-Tracking StudyProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644425(202-213)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644425
Castner NArsiwala-Scheppach LMertens SKrois JThaqi EKasneci EWahl SSchwendicke F(2024)Expert gaze as a usability indicator of medical AI decision support systems: a preliminary studynpj Digital Medicine10.1038/s41746-024-01192-87:1Online publication date: 27-Jul-2024
https://doi.org/10.1038/s41746-024-01192-8
Thilderkvist EDobslaw F(2024)On current limitations of online eye-tracking to study the visual processing of source codeInformation and Software Technology10.1016/j.infsof.2024.107502174(107502)Online publication date: Oct-2024
https://doi.org/10.1016/j.infsof.2024.107502
Cheng GZou DXie HWang F(2024)Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programmingComputers & Education10.1016/j.compedu.2023.104948208:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104948
Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3
Akçay AAltun A(2024)The investigation of human attention networks on debugging performanceEducation and Information Technologies10.1007/s10639-023-11955-729:3(2679-2695)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s10639-023-11955-7
Pacol CRodrigo MTablatin C(2024)A Comparative Study of High and Low Performing Students’ Visual Effort and Attention When Identifying Syntax ErrorsAugmented Cognition10.1007/978-3-031-61569-6_6(77-94)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61569-6_6
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

An eye-tracking study assessing the comprehension of c++ and Python source code

An Eye Tracking Study on camelCase and under_score Identifier Styles

Can the E-Z Reader Model Predict Eye Movements Over Code? Towards a Model of Eye Movements Over Source Code