More Web Proxy on the site http://driver.im/

short-paper

Learning to accelerate compiler testing

Author:

Junjie ChenAuthors Info & Claims

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

Pages 472 - 475

https://doi.org/10.1145/3183440.3183456

Published: 27 May 2018 Publication History

Abstract

Compilers are one of the most important software infrastructures. Compiler testing is an effective and widely-used way to assure the quality of compilers. While many compiler testing techniques have been proposed to detect compiler bugs, these techniques still suffer from the serious efficiency problem. This is because these techniques need to run a large number of randomly generated test programs on the fly through automated test-generation tools (e.g., Csmith). To accelerate compiler testing, it is desirable to schedule the execution order of the generated test programs so that the test programs that are more likely to trigger compiler bugs are executed earlier. Since different test programs tend to trigger the same compiler bug, the ideal goal of accelerating compiler testing is to execute the test programs triggering different compiler bugs in the beginning. However, such perfect goal is hard to achieve, and thus in this work, we design four steps to approach the ideal goal through learning, in order to largely accelerate compiler testing.

References

[1]

Junjie Chen, Yanwei Bai, Dan Hao, Yingfei Xiong, Hongyu Zhang, and Bing Xie. 2017. Learning to prioritize test programs for compiler testing. In ICSE. 700--711.

Digital Library

[2]

Junjie Chen, Yanwei Bai, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, and Bing Xie. 2016. Test case prioritization for compilers: A text-vector based approach. In ICST. 266--277.

[3]

Junjie Chen, Yanwei Bai, Dan Hao, Lingming Zhang, Lu Zhang, and Bing Xie. 2017. How Do Assertions Impact Coverage-Based Test-Suite Reduction?. In ICST. 418--423.

[4]

Junjie Chen, Yanwei Bai, Dan Hao, Lingming Zhang, Lu Zhang, Bing Xie, and Hong Mei. 2016. Supporting oracle construction via static analysis. In ASE. 178--189.

Digital Library

[5]

Junjie Chen, Wenxiang Hu, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, and Bing Xie. 2016. An empirical comparison of compiler testing techniques. In ICSE. 180--190.

Digital Library

[6]

Yang Chen, Alex Groce, Chaoqiang Zhang, Weng-Keen Wong, Xiaoli Fern, Eric Eide, and John Regehr. 2013. Taming Compiler Fuzzers. In PLDI. 197--208.

Digital Library

[7]

Alastair F. Donaldson, Hugues Evrard, Andrei Lascu, and Paul Thomson. 2017. Automated testing of graphics shader compilers. PACMPL 1, OOPSLA (2017), 93:1--93:29.

Digital Library

[8]

Alastair F Donaldson and Andrei Lascu. 2016. Metamorphic testing for (graphics) compilers. In Proceedings of the 1st International Workshop on Metamorphic Testing. 44--47.

Digital Library

[9]

Bo Jiang and W. K. Chan. 2015. Input-based adaptive randomized test case prioritization: A local beam search approach. JSS 105, C (2015), 91--106.

Digital Library

[10]

Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. In PLDI. 25.

Digital Library

[11]

Vu Le, Chengnian Sun, and Zhendong Su. 2015. Finding Deep Compiler Bugs via Guided Stochastic Program Mutation. In OOPSLA. 386--399.

Digital Library

[12]

Vu Le, Chengnian Sun, and Zhendong Su. 2015. Randomized stress-testing of link-time optimizers. In ISSTA. 327--337.

Digital Library

[13]

Christopher Lidbury, Andrei Lascu, Nathan Chong, and Alastair F. Donaldson. 2015. Many-core Compiler Fuzzing. In PLDI. 65--76.

Digital Library

[14]

William M McKeeman. 1998. Differential testing for software. Digital Technical Journal 10, 1 (1998), 100--107.

[15]

John Regehr, Yang Chen, Pascal Cuoq, Eric Eide, Chucky Ellison, and Xuejun Yang. 2012. Test-case reduction for C compiler bugs. In PLDI, Vol. 47. 335--346.

Digital Library

[16]

Chengnian Sun, Vu Le, and Zhendong Su. 2016. Finding and analyzing compiler warning defects. In ICSE. 203--213.

Digital Library

[17]

Chengnian Sun, Vu Le, and Zhendong Su. 2016. Finding compiler bugs via live code mutation. In OOPSLA. 849--863.

Digital Library

[18]

Qiuming Tao, Wei Wu, Chen Zhao, and Wuwei Shen. 2010. An automatic testing approach for compiler based on metamorphic testing technique. In APSEC. 270--279.

Digital Library

[19]

Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and understanding bugs in C compilers. In PLDI. 283--294.

Digital Library

[20]

Lingming Zhang, Dan Hao, Lu Zhang, Gregg Rothermel, and Hong Mei. 2013. Bridging the gap between the total and additional test-case prioritization strategies. In ICSE. 192--201.

Digital Library

[21]

Qirun Zhang, Chengnian Sun, and Zhendong Su. 2017. Skeletal Program Enumeration for Rigorous Compiler Testing. In PLDI. 347--361.

Digital Library

Cited By

Li YDang XMa LKlein JLe Traon YBissyandé T(2024)Test Input Prioritization for 3D Point CloudsACM Transactions on Software Engineering and Methodology10.1145/364367633:5(1-44)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3643676
Dang XLi YPapadakis MKlein JBissyandé TLe Traon Y(2023)GraphPrior: Mutation-based Test Input Prioritization for Graph Neural NetworksACM Transactions on Software Engineering and Methodology10.1145/360719133:1(1-40)Online publication date: 4-Jul-2023
https://dl.acm.org/doi/10.1145/3607191
Chen ZChen JWang WZhou JWang MChen XZhou SWang J(2023)Exploring Better Black-Box Test Case Prioritization via Log AnalysisACM Transactions on Software Engineering and Methodology10.1145/356993232:3(1-32)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3569932
Show More Cited By

Index Terms

Learning to accelerate compiler testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Compilers

Recommendations

Boosting Compiler Testing via Compiler Optimization Exploration
Compilers are a kind of important software, and similar to the quality assurance of other software, compiler testing is one of the most widely-used ways of guaranteeing their quality. Compiler bugs tend to occur in compiler optimizations. Detecting ...
A Survey of Compiler Testing

Virtually any software running on a computer has been processed by a compiler or a compiler-like tool. Because compilers are such a crucial piece of infrastructure for building software, their correctness is of paramount importance. To validate and ...
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '18: Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings

May 2018

231 pages

ISBN:9781450356633

DOI:10.1145/3183440

Conference Chair:
Michel Chaudron
Chalmers University of Technology, University of Gothenburg, Sweden
,
General Chair:
Ivica Crnkovic
Chalmers University of Technology, University of Gothenburg, Sweden
,
Program Chairs:
Marsha Chechik
University of Toronto, Canada
,
Mark Harman
Facebook and University College London, United Kingdom

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

National Natural Science Foundation of China

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 27 - June 3, 2018

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
341
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)5

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YDang XMa LKlein JLe Traon YBissyandé T(2024)Test Input Prioritization for 3D Point CloudsACM Transactions on Software Engineering and Methodology10.1145/364367633:5(1-44)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3643676
Dang XLi YPapadakis MKlein JBissyandé TLe Traon Y(2023)GraphPrior: Mutation-based Test Input Prioritization for Graph Neural NetworksACM Transactions on Software Engineering and Methodology10.1145/360719133:1(1-40)Online publication date: 4-Jul-2023
https://dl.acm.org/doi/10.1145/3607191
Chen ZChen JWang WZhou JWang MChen XZhou SWang J(2023)Exploring Better Black-Box Test Case Prioritization via Log AnalysisACM Transactions on Software Engineering and Methodology10.1145/356993232:3(1-32)Online publication date: 26-Apr-2023
https://dl.acm.org/doi/10.1145/3569932
Chen JSuo C(2022)Boosting Compiler Testing via Compiler Optimization ExplorationACM Transactions on Software Engineering and Methodology10.1145/350836231:4(1-33)Online publication date: 22-Aug-2022
https://dl.acm.org/doi/10.1145/3508362
Zhou JChen JHao D(2021)Parallel Test PrioritizationACM Transactions on Software Engineering and Methodology10.1145/347190631:1(1-50)Online publication date: 28-Sep-2021
https://dl.acm.org/doi/10.1145/3471906
Chen JWang GHao DXiong YZhang HZhang LXie B(2021)Coverage Prediction for Accelerating Compiler TestingIEEE Transactions on Software Engineering10.1109/TSE.2018.288977147:2(261-278)Online publication date: 1-Feb-2021
https://dl.acm.org/doi/10.1109/TSE.2018.2889771
Wang ZYou HChen JZhang YDong XZhang W(2021)Prioritizing Test Inputs for Deep Neural Networks via Mutation AnalysisProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00046(397-409)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00046
Yilmaz FSridhar MChoi W(2020)Guide Me to Exploit: Assisted ROP Exploit Generation for ActionScript Virtual MachineProceedings of the 36th Annual Computer Security Applications Conference10.1145/3427228.3427568(386-400)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3427228.3427568
Chen JWu ZWang ZYou HZhang LYan M(2020)Practical Accuracy Estimation for Efficient Deep Neural Network TestingACM Transactions on Software Engineering and Methodology10.1145/339411229:4(1-35)Online publication date: 4-Oct-2020
https://dl.acm.org/doi/10.1145/3394112
Chen JPatra JPradel MXiong YZhang HHao DZhang L(2020)A Survey of Compiler TestingACM Computing Surveys10.1145/336356253:1(1-36)Online publication date: 6-Feb-2020
https://dl.acm.org/doi/10.1145/3363562
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents