More Web Proxy on the site http://driver.im/

research-article

On reliability of patch correctness assessment

Authors:

Xuan-Bach D. Le,

Corina PasareanuAuthors Info & Claims

ICSE '19: Proceedings of the 41st International Conference on Software Engineering

Pages 524 - 535

https://doi.org/10.1109/ICSE.2019.00064

Published: 25 May 2019 Publication History

Abstract

Current state-of-the-art automatic software repair (ASR) techniques rely heavily on incomplete specifications, or test suites, to generate repairs. This, however, may cause ASR tools to generate repairs that are incorrect and hard to generalize. To assess patch correctness, researchers have been following two methods separately: (1) Automated annotation, wherein patches are automatically labeled by an independent test suite (ITS) - a patch passing the ITS is regarded as correct or generalizable, and incorrect otherwise, (2) Author annotation, wherein authors of ASR techniques manually annotate the correctness labels of patches generated by their and competing tools. While automated annotation cannot ascertain that a patch is actually correct, author annotation is prone to subjectivity. This concern has caused an on-going debate on the appropriate ways to assess the effectiveness of numerous ASR techniques proposed recently.

In this work, we propose to assess reliability of author and automated annotations on patch correctness assessment. We do this by first constructing a gold set of correctness labels for 189 randomly selected patches generated by 8 state-of-the-art ASR techniques through a user study involving 35 professional developers as independent annotators. By measuring inter-rater agreement as a proxy for annotation quality - as commonly done in the literature - we demonstrate that our constructed gold set is on par with other high-quality gold sets. We then compare labels generated by author and automated annotations with this gold set to assess reliability of the patch assessment methodologies. We subsequently report several findings and highlight implications for future studies.

References

[1]

G. Tassey, "The economic impacts of inadequate infrastructure for software testing." Planning Report, NIST, 2002.

[2]

T. Britton, L. Jeng, G. Carver, P. Cheak, and T. Katzenellenbogen, "Reversible debugging software," University of Cambridge, Judge Business School, Tech. Rep., 2013.

[3]

S. Mechtaev, J. Yi, and A. Roychoudhury, "Directrix: Looking for simple program repairs," in International Conference on Software Engineering (ICSE). IEEE Press, 2015, pp. 448--458.

Digital Library

[4]

S. Mechtaev, "Angelix: Scalable multiline program patch synthesis via symbolic analysis," in International Conference on Software Engineering (ICSE). IEEE, 2016, pp. 691--701.

Digital Library

[5]

Y. Xiong, J. Wang, R. Yan, J. Zhang, S. Han, G. Huang, and L. Zhang, "Precise condition synthesis for program repair," in International Conference on Software Engineering (ICSE). IEEE Press, 2017, pp. 416--426.

Digital Library

[6]

F. Long and M. Rinard, "Staged program repair with condition synthesis," in European Software Engineering Conference and International Symposium on Foundations of Software Engineering (ESEC/FSE), 2015, pp. 166--178.

Digital Library

[7]

F. Long, "Automatic patch generation by learning correct code," in Symposium on Principles of Programming Languages (POPL), 2016, pp. 298--312.

Digital Library

[8]

J. Xuan, M. Martinez, F. Demarco, M. Clément, S. Lamelas, T. Durieux, D. Le Berre, and M. Monperrus, "Nopol: Automatic repair of conditional statement bugs in java programs," Transactions on Software Engineering, 2016.

Digital Library

[9]

C. Le Goues, M. Dewey-Vogt, S. Forrest, and W. Weimer, "A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each," in International Conference on Software Engineering, ser. ICSE'12, 2012, pp. 3--13.

Digital Library

[10]

D. Kim, J. Nam, J. Song, and S. Kim, "Automatic patch generation learned from human-written patches," in International Conference on Software Engineering, ser. ICSE '13, 2013, pp. 802--811.

Digital Library

[11]

X.-B. D. Le, T.-D. B. Le, and D. Lo, "Should fixing these failures be delegated to automated program repair?" in International Symposium on Software Reliability Engineering (ISSRE), 2015, pp. 427--437.

Digital Library

[12]

X. B. D. Le, Q. L. Le, D. Lo, and C. Le Goues, "Enhancing automated program repair with deductive verification," in International Conference on Software Maintenance and Evolution (ICSME), 2016, pp. 428--432.

[13]

X.-B. D. Le, D.-H. Chu, D. Lo, C. Le Goues, and W. Visser, "Jfix: Semantics-based repair of java programs via symbolic pathfinder," in International Symposium on Software Testing and Analysis, ser. ISSTA'17, 2017 (to appear).

Digital Library

[14]

S. Chandra, E. Torlak, S. Barman, and R. Bodik, "Angelic debugging," in International Conference on Software Engineering, ser. ICSE'11, 2011, pp. 121--130.

Digital Library

[15]

Z. Qi, F. Long, S. Achour, and M. Rinard, "An analysis of patch plausibility and correctness for generate-and-validate patch generation systems," in International Symposium on Software Testing and Analysis. ACM, 2015, pp. 24--36.

Digital Library

[16]

F. Long and M. Rinard, "An analysis of the search spaces for generate and validate patch generation systems," in International Conference on Software Engineering (ICSE). ACM, 2016, pp. 702--713.

Digital Library

[17]

E. K. Smith, E. T. Barr, C. Le Goues, and Y. Brun, "Is the cure worse than the disease? overfitting in automated program repair," in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 2015, pp. 532--543.

Digital Library

[18]

X.-B. D. Le, F. Thung, D. Lo, and C. L. Goues, "Overfitting in semantics-based automated program repair," in Proceedings of the 40th International Conference on Software Engineering, ser. ICSE '18, 2018, pp. 163--163.

Digital Library

[19]

X.-B. D. Le, D. Lo, and C. Le Goues, "Empirical study on synthesis engines for semantics-based program repair," in International Conference on Software Maintenance and Evolution, ser. ICSME'16, 2016, pp. 423--427.

[20]

Y. Xiong, J. Wang, R. Yan, J. Zhang, S. Han, G. Huang, and L. Zhang, "Precise condition synthesis for program repair," in International Conference on Software Engineering. IEEE Press, 2017, pp. 416--426.

Digital Library

[21]

Y. Xiong, X. Liu, M. Zeng, L. Zhang, and G. Huang, "Identifying patch correctness in test-based program repair," in Proceedings of the 40th International Conference on Software Engineering. ACM, 2018, pp. 789--799.

Digital Library

[22]

X. B. D. Le, D. H. Chu, D. Lo, C. Le Goues, and W. Visser, "S3: syntax-and semantic-guided repair synthesis via programming by example," FSE. ACM, 2017.

Digital Library

[23]

R. Just, D. Jalali, and M. D. Ernst, "Defects4j: A database of existing faults to enable controlled testing studies for java programs," in International Symposium on Software Testing and Analysis, ser. ISSTA '14, 2014, pp. 437--440.

Digital Library

[24]

M. Martinez, T. Durieux, R. Sommerard, J. Xuan, and M. Monperrus, "Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset," Empirical Software Engineering, vol. 22, no. 4, pp. 1936--1964, 2017. {Online}. Available

Digital Library

[25]

Q. Xin and S. P. Reiss, "Identifying test-suite-overfitted patches through test case generation," in International Symposium on Software Testing and Analysis. ACM, 2017, pp. 226--236.

Digital Library

[26]

C. Pacheco, S. K. Lahiri, M. D. Ernst, and T. Ball, "Feedback-directed random test generation," in 29th International Conference on Software Engineering (ICSE 2007), Minneapolis, MN, USA, May 20--26, 2007, 2007, pp. 75--84. {Online}. Available

Digital Library

[27]

D. M. Christopher, R. Prabhakar, and S. Hinrich, "Introduction to information retrieval," An Introduction To Information Retrieval, vol. 151, p. 177, 2008.

Digital Library

[28]

T. T. Damessie, T. P. Nghiem, F. Scholer, and J. S. Culpepper, "Gauging the quality of relevance assessments using inter-rater agreement," in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, August 7--11, 2017, 2017, pp. 1089--1092.

Digital Library

[29]

S. Jha, S. Gulwani, S. A. Seshia, and A. Tiwari, "Oracle-guided component-based program synthesis," in International Conference on Software Engineering (ICSE), Cape Town, South Africa, 2010, pp. 215--224.

Digital Library

[30]

R. Alur, R. Bodik, G. Juniwal, M. M. Martin, M. Raghothaman, S. A. Seshia, R. Singh, A. Solar-Lezama, E. Torlak, and A. Udupa, "Syntax-guided synthesis," Dependable Software Systems Engineering, 2015.

[31]

W. Weimer, Z. P. Fry, and S. Forrest, "Leveraging program equivalence for adaptive program repair: Models and first results," in Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering. IEEE Press, 2013, pp. 356--366.

Digital Library

[32]

Y. Qi, X. Mao, Y. Lei, Z. Dai, and C. Wang, "The strength of random search on automated program repair," in Proceedings of the 36th International Conference on Software Engineering. ACM, 2014, pp. 254--265.

Digital Library

[33]

C. Cadar, D. Dunbar, D. R. Engler et al., "Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs." in Symposium on Operating Systems Design and Implementation (OSDI), 2008, pp. 209--224.

Digital Library

[34]

X. B. D. Le, D. Lo, and C. Le Goues, "History driven program repair," in International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, 2016, pp. 213--224.

[35]

L. Dybkjaer, H. Hemsen, and W. Minker, Evaluation of Text and Speech Systems, 1st ed. Springer Publishing Company, Incorporated, 2007.

Digital Library

[36]

S. Rastkar, G. C. Murphy, and G. Murray, "Summarizing software artifacts: a case study of bug reports," in Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering-Volume 1. ACM, 2010, pp. 505--514.

Digital Library

[37]

D. D. Gachechiladze, F. Lanubile, N. Novielli, and A. Serebrenik, "Anger and its direction in apache jira developer comments," in Proc. of the Int. Conf. on Software Engineering (ICSE), 2017.

Digital Library

[38]

R. P. Buse and W. R. Weimer, "Learning a metric for code readability," IEEE Transactions on Software Engineering, vol. 36, no. 4, pp. 546--558, 2010.

Digital Library

[39]

A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella, and S. Panichella, "Labeling source code with information retrieval methods: an empirical study," Empirical Software Engineering, pp. 1383--1420, 2014.

Digital Library

[40]

Y. Zou, T. Ye, Y. Lu, J. Mylopoulos, and L. Zhang, "Learning to rank for question-oriented software text retrieval (t)," in Automated Software Engineering (ASE), 2015 30th IEEE/ACM International Conference on. IEEE, 2015, pp. 1--11.

Digital Library

[41]

O. Ormandjieva, I. Hussain, and L. Kosseim, "Toward a text classification system for the quality assessment of software requirements written in natural language," in Fourth international workshop on Software quality assurance: in conjunction with the 6th ESEC/FSE joint meeting. ACM, 2007, pp. 39--45.

Digital Library

[42]

C. Treude, M. P. Robillard, and B. Dagenais, "Extracting development tasks to navigate software documentation," IEEE Transactions on Software Engineering, vol. 41, no. 6, pp. 565--581, 2015.

Digital Library

[43]

F. Scholer, A. Turpin, and M. Sanderson, "Quantifying test collection quality based on the consistency of relevance judgements," in Proceeding of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2011, Beijing, China, July 25--29, 2011, 2011, pp. 1063--1072.

Digital Library

[44]

J. Cohen, "A coefficient of agreement for nominal scales," Educational and psychological measurement, vol. 20, no. 1, pp. 37--46, 1960.

[45]

K. Krippendorff, "Estimating the reliability, systematic error, and random error of interval data," Educational and Psychological Measurement, vol. 30, no. 1, pp. 61--70, 1970.

[46]

J. R. Landis and G. G. Koch, "The measurement of observer agreement for categorical data," biometrics, pp. 159--174, 1977.

[47]

N. Cliff, "Dominance statistics: Ordinal analyses to answer ordinal questions," Psychological Bulletin, vol. 114, no. 3, p. 494, 1993.

[48]

A. R. Vaccaro, A. Patel, and C. Fisher, "Author conflict and bias in research: Quantifying the downgrade in methodology," Spine, vol. 30, no. 14, 2011.

[49]

C. D. Wickens, "Processing resources and attention," Multiple-task performance, vol. 1991, pp. 3--34, 1991.

[50]

G. Fraser and A. Arcuri, "Evosuite: automatic test suite generation for object-oriented software," in SIGSOFT/FSE'11 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-19) and ESEC'11: 13th European Software Engineering Conference (ESEC-13), 2011, pp. 416--419.

Digital Library

[51]

M. Monperrus, "A critical review of automatic patch generation learned from human-written patches: essay on the problem statement and the evaluation of automatic software repair," in Proceedings of the 36th International Conference on Software Engineering. ACM, 2014, pp. 234--242.

Digital Library

[52]

M. Sipser, Introduction to the Theory of Computation, 1st ed. International Thomson Publishing, 1996.

Digital Library

[53]

F. Thung, D. Lo, and L. Jiang, "Automatic defect categorization," in Reverse Engineering (WCRE), 2012 19th Working Conference on. IEEE, 2012, pp. 205--214.

Digital Library

[54]

A. Bacchelli, T. Dal Sasso, M. D'Ambros, and M. Lanza, "Content classification of development emails," in Proceedings of the 34th International Conference on Software Engineering. IEEE Press, 2012, pp. 375--385.

Digital Library

[55]

F. Thung, X.-B. D. Le, and D. Lo, "Active semi-supervised defect categorization," in Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension. IEEE Press, 2015, pp. 60--70.

Digital Library

[56]

C. Bird, A. Bachmann, E. Aune, J. Duffy, A. Bernstein, V. Filkov, and P. T. Devanbu, "Fair and balanced?: bias in bug-fix datasets," in Proceedings of the 7th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2009, Amsterdam, The Netherlands, August 24--28, 2009, 2009, pp. 121--130.

Digital Library

[57]

P. S. Kochhar, Y. Tian, and D. Lo, "Potential biases in bug localization: Do they matter?" in Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, 2014, pp. 803--814.

Digital Library

[58]

C. Bird, "Dont embarrass yourself: Beware of bias in your data," in Perspectives on Data Science for Software Engineering. Elsevier, 2016, pp. 309--315.

[59]

C. Parnin and A. Orso, "Are automated debugging techniques actually helping programmers?" in Proceedings of the 2011 International Symposium on Software Testing and Analysis. ACM, 2011, pp. 199--209.

Digital Library

[60]

X.-B. D. Le, "Towards efficient and effective automatic program repair," in Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, ser. ASE 2016, 2016, pp. 876--879.

Digital Library

[61]

D. R. Cok, "Openjml: Jml for java 7 by extending openjdk," in NASA Formal Methods Symposium. Springer, 2011, pp. 472--479.

Digital Library

[62]

G. T. Leavens, A. L. Baker, and C. Ruby, "Jml: a java modeling language," in Formal Underpinnings of Java Workshop (at OOPSLA98), 1998, pp. 404--420.

[63]

O. Baysal, R. Holmes, and M. W. Godfrey, "No issue left behind: Reducing information overload in issue tracking," in Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 2014, pp. 666--677.

Digital Library

[64]

A. J. Ko, B. Dosono, and N. Duriseti, "Thirty years of software problems in the news," in Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering. ACM, 2014, pp. 32--39.

Digital Library

[65]

E. Daka, J. Campos, G. Fraser, J. Dorn, and W. Weimer, "Modeling readability to improve unit tests," in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. ACM, 2015, pp. 107--118.

Digital Library

[66]

P. Bailey, N. Craswell, I. Soboroff, P. Thomas, A. P. de Vries, and E. Yilmaz, "Relevance assessment: are judges exchangeable and does it matter," in Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 2008, pp. 667--674.

Digital Library

[67]

X.-B. D. Le, Dataset, 2009. {Online}. Available: https://github.com/anonymousICSE2019/patchcorrectness

[68]

K. Kevic, B. M. Walters, T. R. Shaffer, B. Sharif, D. C. Shepherd, and T. Fritz, "Tracing software developers' eyes and interactions for change tasks," in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2015, Bergamo, Italy, August 30 - September 4, 2015, 2015, pp. 202--213.

Digital Library

[69]

B. Johnson, Y. Song, E. Murphy-Hill, and R. Bowdidge, "Why don't software developers use static analysis tools to find bugs?" in Software Engineering (ICSE), 2013 35th International Conference on. IEEE, 2013, pp. 672--681.

Digital Library

[70]

J. Rubin and M. Rinard, "The challenges of staying together while moving fast: An exploratory study," in Software Engineering (ICSE), 2016 IEEE/ACM 38th International Conference on. IEEE, 2016, pp. 982--993.

Digital Library

[71]

C. Castillo, D. Donato, L. Becchetti, P. Boldi, S. Leonardi, M. Santini, and S. Vigna, "A reference collection for web spam," in ACM Sigir Forum, vol. 40, no. 2. ACM, 2006, pp. 11--24.

Digital Library

[72]

E. Meij, "Combining concepts and language models for information access," in SIGIR Forum, vol. 45, no. 1, 2011, p. 80.

Digital Library

[73]

E. Amigó, J. Gonzalo, and F. Verdejo, "A general evaluation measure for document organization tasks," in Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval. ACM, 2013, pp. 643--652.

Digital Library

[74]

O. Chaparro, J. Lu, F. Zampetti, L. Moreno, M. Di Penta, A. Marcus, G. Bavota, and V. Ng, "Detecting missing information in bug descriptions," in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 2017, pp. 396--407.

Digital Library

[75]

R. Abdalkareem, O. Nourry, S. Wehaibi, S. Mujahid, and E. Shihab, "Why do developers use trivial packages? an empirical case study on npm," in Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, 2017, pp. 385--395.

Digital Library

[76]

Y. Qi, X. Mao, Y. Lei, Z. Dai, and C. Wang, "The strength of random search on automated program repair," in International Conference on Software Engineering (ICSE). ACM, 2014, pp. 254--265.

Digital Library

[77]

H. D. T. Nguyen, D. Qi, A. Roychoudhury, and S. Chandra, "Semfix: Program repair via semantic analysis," in International Conference on Software Engineering (ICSE). IEEE Press, 2013, pp. 772--781.

Digital Library

[78]

L. D'Antoni, R. Samanta, and R. Singh, "Qlose: Program repair with quantitative objectives," in International Conference on Computer Aided Verification (CAV). Springer, 2016, pp. 383--401.

[79]

R. K. Saha, Y. Lyu, H. Yoshida, and M. R. Prasad, "Elixir: Effective object-oriented program repair," in Automated Software Engineering (ASE), 2017 32nd IEEE/ACM International Conference on. IEEE, 2017, pp. 648--659.

Digital Library

[80]

L. Chen, Y. Pei, and C. A. Furia, "Contract-based program repair without the contracts," in Automated Software Engineering (ASE), 2017 32nd IEEE/ACM International Conference on. IEEE, 2017, pp. 637--647.

Digital Library

[81]

L. Gazzola, D. Micucci, and L. Mariani, "Automatic software repair: A survey," IEEE Transactions on Software Engineering, 2017.

Digital Library

[82]

B. Carterette and I. Soboroff, "The effect of assessor error on ir system evaluation," in Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, 2010, pp. 539--546.

Digital Library

[83]

C. Le Goues, N. Holtschulte, E. K. Smith, Y. Brun, P. Devanbu, S. Forrest, and W. Weimer, "The ManyBugs and IntroClass benchmarks for automated repair of C programs," Transactions on Software Engineering (TSE), vol. 41, no. 12, pp. 1236--1256, Dec. 2015.

[84]

K. Herzig, S. Just, and A. Zeller, "It's not a bug, it's a feature: how misclassification impacts bug prediction," in 35th International Conference on Software Engineering, ICSE '13, San Francisco, CA, USA, May 18--26, 2013, 2013, pp. 392--401.

Digital Library

Cited By

Ye ZZhou LZhang FJin WNing ZHu YQin ZChristakis MPradel M(2024)FortifyPatch: Towards Tamper-Resistant Live Patching in Linux-Based HypervisorProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652108(38-50)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652108
Yang WSong LXue YRoychoudhury APaiva AAbreu RStorey M(2024)Rust-lancet: Automated Ownership-Rule-Violation Fixing with Behavior PreservationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639103(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639103
Eladawy HLe Goues CBrun YRoychoudhury APaiva AAbreu RStorey M(2024)Automated Program Repair, What Is It Good For? Not Absolutely Nothing!Proceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639095(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639095
Show More Cited By

Recommendations

Automated patch correctness assessment: how far are we?
ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering

Test-based automated program repair (APR) has attracted huge attention from both industry and academia. Despite the significant progress made in recent studies, the overfitting problem (i.e., the generated patch is plausible but overfitting) is still a ...
Patch correctness assessment in automated program repair based on the impact of patches on production and test code
ISSTA 2022: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis

Test-based generate-and-validate automated program repair (APR) systems often generate many patches that pass the test suite without fixing the bug. The generated patches must be manually inspected by the developers, so previous research proposed ...
Context-Aware Code Change Embedding for Better Patch Correctness Assessment
Despite the capability in successfully fixing more and more real-world bugs, existing Automated Program Repair (APR) techniques are still challenged by the long-standing overfitting problem (i.e., a generated patch that passes all tests is actually ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '19: Proceedings of the 41st International Conference on Software Engineering

May 2019

1318 pages

General Chair:
Joanne M. Atlee
University of Waterloo, Canada
,
Program Chairs:
Tevfik Bultan
University of California, Santa Barbara
,
Jon Whittle
Monash University, Australia

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

IEEE Press

Publication History

Published: 25 May 2019

Check for updates

Qualifiers

Research-article

Conference

ICSE '19

Sponsor:

SIGSOFT
IEEE-CS

ICSE '19: 41st International Conference on Software Engineering

May 25 - 31, 2019

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
199
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ye ZZhou LZhang FJin WNing ZHu YQin ZChristakis MPradel M(2024)FortifyPatch: Towards Tamper-Resistant Live Patching in Linux-Based HypervisorProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652108(38-50)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652108
Yang WSong LXue YRoychoudhury APaiva AAbreu RStorey M(2024)Rust-lancet: Automated Ownership-Rule-Violation Fixing with Behavior PreservationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639103(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639103
Eladawy HLe Goues CBrun YRoychoudhury APaiva AAbreu RStorey M(2024)Automated Program Repair, What Is It Good For? Not Absolutely Nothing!Proceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639095(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639095
Zhang QFang CSun WLiu YHe THao XChen Z(2024)APPT: Boosting Automated Patch Correctness Prediction via Fine-Tuning Pre-Trained ModelsIEEE Transactions on Software Engineering10.1109/TSE.2024.335496950:3(474-494)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3354969
Zhang QFang CMa YSun WChen Z(2023)A Survey of Learning-based Automated Program RepairACM Transactions on Software Engineering and Methodology10.1145/363197433:2(1-69)Online publication date: 23-Dec-2023
https://dl.acm.org/doi/10.1145/3631974
Yang JWang YLou YWen MZhang LChandra SBlincoe KTonella P(2023)A Large-Scale Empirical Review of Patch Correctness Checking ApproachesProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616331(1203-1215)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616331
Nilizadeh ALeavens GPăsăreanu CNoller Y(2023)JMLKelinci+: Detecting Semantic Bugs and Covering Branches with Valid Inputs Using Coverage-guided Fuzzing and Runtime Assertion CheckingFormal Aspects of Computing10.1145/360753836:1(1-24)Online publication date: 5-Aug-2023
https://dl.acm.org/doi/10.1145/3607538
Renzullo JWeimer WForrest S(2023)Evolving Software: Combining Online Learning with Mutation-Based Stochastic SearchACM Transactions on Evolutionary Learning and Optimization10.1145/35976173:4(1-32)Online publication date: 23-May-2023
https://dl.acm.org/doi/10.1145/3597617
Liu KZhang JLi LKoyuncu AKim DGe CLiu ZKlein JBissyandé T(2023)Reliable Fix Patterns Inferred from Static Checkers for Automated Program RepairACM Transactions on Software Engineering and Methodology10.1145/357963732:4(1-38)Online publication date: 26-May-2023
https://dl.acm.org/doi/10.1145/3579637
Tian HLiu KLi YKaboré AKoyuncu AHabib ALi LWen JKlein JBissyandé T(2023)The Best of Both Worlds: Combining Learned Embeddings with Engineered Features for Accurate Prediction of Correct PatchesACM Transactions on Software Engineering and Methodology10.1145/357603932:4(1-34)Online publication date: 27-May-2023
https://dl.acm.org/doi/10.1145/3576039
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents