More Web Proxy on the site http://driver.im/

research-article

Open access

Guiding Greybox Fuzzing with Mutation Testing

Authors:

Vasudev Vikram,

Isabella Laybourn,

Rafaello Sanna,

Rohan PadhyeAuthors Info & Claims

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Pages 929 - 941

https://doi.org/10.1145/3597926.3598107

Published: 13 July 2023 Publication History

Abstract

Greybox fuzzing and mutation testing are two popular but mostly independent fields of software testing research that have so far had limited overlap. Greybox fuzzing, generally geared towards searching for new bugs, predominantly uses code coverage for selecting inputs to save. Mutation testing is primarily used as a stronger alternative to code coverage in assessing the quality of regression tests; the idea is to evaluate tests for their ability to identify artificially injected faults in the target program. But what if we wanted to use greybox fuzzing to synthesize high-quality regression tests?

In this paper, we develop and evaluate Mu2, a Java-based framework for incorporating mutation analysis in the greybox fuzzing loop, with the goal of producing a test-input corpus with a high mutation score. Mu2 makes use of a differential oracle for identifying inputs that exercise interesting program behavior without causing crashes. This paper describes several dynamic optimizations implemented in Mu2 to overcome the high cost of performing mutation analysis with every fuzzer-generated input. These optimizations introduce trade-offs in fuzzing throughput and mutation killing ability, which we evaluate empirically on five real-world Java benchmarks. Overall, variants of Mu2 are able to synthesize test-input corpora with a higher mutation score than state-of-the-art Java fuzzer Zest.

References

[1]

Paul Ammann. 2015. Transforming mutation testing from the technology of the future into the technology of the present. In International conference on software testing, verification and validation workshops (ICST): Mutation workshop. IEEE. https://mutation-workshop.github.io/2015/program/MutationKeynote.pdf

[2]

J. H. Andrews, L. C. Briand, and Y. Labiche. 2005. Is Mutation an Appropriate Tool for Testing Experiments? In Proceedings of the 27th International Conference on Software Engineering (ICSE ’05). Association for Computing Machinery, 402–411. isbn:1581139632 https://doi.org/10.1145/1062455.1062530

Digital Library

[3]

Apache Foundation. 2022. Tomcat. https://github.com/apache/tomcat Retrieved August 31, 2022

[4]

Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, and Daniel Teuchert. 2019. Nautilus: Fishing for Deep Bugs with Grammars. In 26th Annual Network and Distributed System Security Symposium (NDSS ’19). https://doi.org/10.14722/ndss.2019.23412

[5]

Cornelius Aschermann, Sergej Schumilo, Tim Blazytko, Robert Gawlik, and Thorsten Holz. 2019. REDQUEEN: Fuzzing with Input-to-State Correspondence. In NDSS. 19, 1–15. https://doi.org/10.14722/ndss.2019.23371

[6]

Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2015. The Oracle Problem in Software Testing: A Survey. IEEE Transactions on Software Engineering, 41, 5 (2015), 507–525. https://doi.org/10.1109/TSE.2014.2372785

Digital Library

[7]

U. C. Berkeley. 2019. ChocoPy. https://chocopy.org/ Reference compiler JAR retrieved on January 12, 2022

[8]

Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2329–2344. https://doi.org/10.1145/3133956.3134020

Digital Library

[9]

Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage-based Greybox Fuzzing as Markov Chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS). 1032–1043. https://doi.org/10.1145/2976749.2978428

Digital Library

[10]

Marcel Böhme, László Szekeres, and Jonathan Metzman. 2022. On the Reliability of Coverage-Based Fuzzer Benchmarking. In 44th IEEE/ACM International Conference on Software Engineering (ICSE’22). https://doi.org/10.1145/3510003.3510230 to appear

Digital Library

[11]

Sang Kil Cha, Maverick Woo, and David Brumley. 2015. Program-adaptive mutational fuzzing. In 2015 IEEE Symposium on Security and Privacy. 725–741. https://doi.org/10.1109/SP.2015.50

Digital Library

[12]

Thierry Titcheu Chekam, Mike Papadakis, Yves Le Traon, and Mark Harman. 2017. An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). 597–608. https://doi.org/10.1109/ICSE.2017.61

Digital Library

[13]

Hongxu Chen, Yinxing Xue, Yuekang Li, Bihuan Chen, Xiaofei Xie, Xiuheng Wu, and Yang Liu. 2018. Hawkeye: Towards a desired directed grey-box fuzzer. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2095–2108. https://doi.org/10.1145/3243734.3243849

Digital Library

[14]

Peng Chen and Hao Chen. 2018. Angora: Efficient fuzzing by principled search. In 2018 IEEE Symposium on Security and Privacy (SP). 711–725. https://doi.org/10.1109/SP.2018.00046

[15]

Yiqun T. Chen, Rahul Gopinath, Anita Tadakamalla, Michael D. Ernst, Reid Holmes, Gordon Fraser, Paul Ammann, and René Just. 2020. Revisiting the Relationship between Fault Detection, Test Adequacy Criteria, and Test Set Size. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (ASE ’20). Association for Computing Machinery, 237–249. isbn:9781450367684 https://doi.org/10.1145/3324884.3416667

Digital Library

[16]

Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In Proceedings of the 5th ACM SIGPLAN International Conference on Functional Programming (ICFP). https://doi.org/10.1145/351240.351266

Digital Library

[17]

Henry Coles, Thomas Laurent, Christopher Henard, Mike Papadakis, and Anthony Ventresque. 2016. PIT: a practical mutation testing tool for Java. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA’16). 449–452. https://doi.org/10.1145/2931037.2948707

Digital Library

[18]

Fabiano Cutigi Ferrari, Alessandro Viola Pizzoleto, and Jeff Offutt. 2018. A Systematic Review of Cost Reduction Techniques for Mutation Testing: Preliminary Results. In 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). 1–10. https://doi.org/10.1109/ICSTW.2018.00021

[19]

R.A. DeMillo, R.J. Lipton, and F.G. Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer, 11, 4 (1978), 34–41. https://doi.org/10.1109/C-M.1978.218136

Digital Library

[20]

Zhen Yu Ding and Claire Le Goues. 2021. An Empirical Study of OSS-Fuzz Bugs. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR). 131–142. https://doi.org/10.1109/MSR52588.2021.00026

[21]

Robert B Evans and Alberto Savoia. 2007. Differential testing: a new approach to change detection. In The 6th Joint Meeting on European software engineering conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering: Companion Papers. 549–552. https://doi.org/10.1145/1295014.1295038

Digital Library

[22]

FasterXML. [n. d.]. Jackson: JSON for Java. https://github.com/FasterXML/jackson Retrieved August 31, 2022

[23]

Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic Test Suite Generation for Object-oriented Software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (ESEC/FSE ’11). https://doi.org/10.1145/2025113.2025179

Digital Library

[24]

Gordon Fraser and Andreas Zeller. 2010. Mutation-driven Generation of Unit Tests and Oracles. In Proceedings of the 19th International Symposium on Software Testing and Analysis (ISSTA ’10). ACM, 147–158. isbn:978-1-60558-823-0 https://doi.org/10.1145/1831708.1831728

Digital Library

[25]

Shuitao Gan, Chao Zhang, Peng Chen, Bodong Zhao, Xiaojun Qin, Dong Wu, and Zuoning Chen. 2020. GREYONE: Data flow sensitive fuzzing. In 29th USENIX Security Symposium (USENIX Security 20). 2577–2594. https://doi.org/10.5555/3489212.3489357

Digital Library

[26]

Ali Ghanbari and Andrian Marcus. 2022. Faster Mutation Analysis with MeMu. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022). 781–784. isbn:9781450393799 https://doi.org/10.1145/3533767.3543288

Digital Library

[27]

John B. Goodenough and Susan L. Gerhart. 1975. Toward a Theory of Test Data Selection. SIGPLAN Not., 10, 6 (1975), apr, 493–510. issn:0362-1340 https://doi.org/10.1145/390016.808473

Digital Library

[28]

Google. 2019. Ideal integration with OSS-Fuzz. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/#regression-testing Retrieved August 31, 2022

[29]

Google. 2021. Gson: A Java serialization/deserialization library to convert Java Objects into JSON and back. https://github.com/google/gson Retrieved August 31, 2022

[30]

Google. 2022. Google Closure Compiler. https://github.com/google/closure-compiler Retrieved August 31, 2022

[31]

Rahul Gopinath, Philipp Görz, and Alex Groce. 2022. Mutation Analysis: Answering the Fuzzing Challenge. CoRR, abs/2201.11303 (2022), arXiv:2201.11303. arxiv:2201.11303

[32]

Rahul Gopinath, Carlos Jensen, and Alex Groce. 2014. Mutations: How Close are they to Real Faults? In 2014 IEEE 25th International Symposium on Software Reliability Engineering. 189–200. https://doi.org/10.1109/ISSRE.2014.40

Digital Library

[33]

Alex Groce, Goutamkumar Tulajappa Kalburgi, Claire Le Goues, Kush Jain, and Rahul Gopinath. 2022. Registered Report: First, Fuzz the Mutants. In International Fuzzing Workshop (FUZZING’22).

[34]

Giovani Guizzo, Federica Sarro, Jens Krinke, and Silvia R. Vergilio. 2022. Sentinel: A Hyper-Heuristic for the Generation of Mutant Reduction Strategies. IEEE Transactions on Software Engineering, 48, 3 (2022), 803–818. https://doi.org/10.1109/TSE.2020.3002496

[35]

William E. Howden. 1982. Weak mutation testing and completeness of test sets. IEEE Transactions on Software Engineering, SE-8, 4 (1982), 371–379. https://doi.org/10.1109/TSE.1982.235571

Digital Library

[36]

Laura Inozemtseva and Reid Holmes. 2014. Coverage is not strongly correlated with test suite effectiveness. In Proceedings of the 36th international conference on software engineering (ICSE’14). 435–445. https://doi.org/10.1145/2568225.2568271

Digital Library

[37]

Yue Jia and Mark Harman. 2010. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering, 37, 5 (2010), 649–678. https://doi.org/10.1109/TSE.2010.62

Digital Library

[38]

René Just, Michael D Ernst, and Gordon Fraser. 2014. Efficient mutation analysis by propagating and partitioning infected execution states. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA’14). 315–326. https://doi.org/10.1145/2610384.2610388

Digital Library

[39]

René Just, Darioush Jalali, Laura Inozemtseva, Michael D Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing? In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’14). 654–665. https://doi.org/10.1145/2635868.2635929

Digital Library

[40]

Samuel J. Kaufman, Ryan Featherman, Justin Alvin, Bob Kurtz, Paul Ammann, and René Just. 2022. Prioritizing Mutants to Guide Mutation Testing. In Proceedings of the 44th International Conference on Software Engineering (ICSE ’22). Association for Computing Machinery, 1743–1754. isbn:9781450392211 https://doi.org/10.1145/3510003.3510187

Digital Library

[41]

Marinos Kintis, Mike Papadakis, Andreas Papadopoulos, Evangelos Valvis, Nicos Malevris, and Yves Le Traon. 2018. How effective are mutation testing tools? An empirical analysis of Java mutation testing tools with manual analysis and real faults. Empirical Software Engineering, 23, 4 (2018), 2426–2463. https://doi.org/10.1007/s10664-017-9582-5

Digital Library

[42]

George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating fuzz testing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2123–2138. https://doi.org/10.1145/3243734.3243804

Digital Library

[43]

James Kukucka, Luís Pina, Paul Ammann, and Jonathan Bell. 2022. CONFETTI: Amplifying Concolic Guidance for Fuzzers. In 44th IEEE/ACM International Conference on Software Engineering (ICSE’22). https://doi.org/10.1145/3510003.3510628

Digital Library

[44]

Leonidas Lampropoulos, Michael Hicks, and Benjamin C Pierce. 2019. Coverage guided, property based testing. Proceedings of the ACM on Programming Languages, 3, OOPSLA (2019), 1–29. https://doi.org/10.1145/3360607

Digital Library

[45]

Thomas Laurent, Mike Papadakis, Marinos Kintis, Christopher Henard, Yves Le Traon, and Anthony Ventresque. 2017. Assessing and improving the mutation testing practice of PIT. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST). 430–435. https://doi.org/10.1109/ICST.2017.47

[46]

Isabella Laybourn. 2022. μ ^2: Using Mutation Analysis to Guide Mutation-Based Fuzzing. In Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings (ICSE ’22). Association for Computing Machinery, 331–333. isbn:9781450392235 https://doi.org/10.1145/3510454.3522682

Digital Library

[47]

Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler Validation via Equivalence modulo Inputs. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’14). 216–226. isbn:9781450327848 https://doi.org/10.1145/2594291.2594334

Digital Library

[48]

Caroline Lemieux and Koushik Sen. 2018. Fairfuzz: A targeted mutation strategy for increasing greybox fuzz testing coverage. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. 475–485. https://doi.org/10.1145/3238147.3238176

Digital Library

[49]

LLVM Compiler Infrastructure. 2016. libFuzzer. https://llvm.org/docs/LibFuzzer.html Accessed February 11, 2022

[50]

Chenyang Lyu, Shouling Ji, Chao Zhang, Yuwei Li, Wei-Han Lee, Yu Song, and Raheem Beyah. 2019. MOPT: Optimized mutation scheduling for fuzzers. In 28th USENIX Security Symposium (USENIX Security 19). 1949–1966. https://doi.org/10.5555/3361338.3361473

Digital Library

[51]

Lech Madeyski, Wojciech Orzeszyna, Richard Torkar, and Mariusz Jozala. 2013. Overcoming the equivalent mutant problem: A systematic literature review and a comparative experiment of second order mutation. IEEE Transactions on Software Engineering, 40, 1 (2013), 23–42. https://doi.org/10.1109/TSE.2013.44

Digital Library

[52]

Valentin Jean Marie Manès, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J Schwartz, and Maverick Woo. 2019. The art, science, and engineering of fuzzing: A survey. IEEE Transactions on Software Engineering, https://doi.org/10.1109/TSE.2019.2946563

[53]

William M. McKeeman. 1998. Differential Testing for Software. DIGITAL TECHNICAL JOURNAL, 10, 1 (1998), 100–107.

[54]

Barton P. Miller, Louis Fredriksen, and Bryan So. 1990. An Empirical Study of the Reliability of UNIX Utilities. Commun. ACM, 33, 12 (1990), dec, 32–44. issn:0001-0782 https://doi.org/10.1145/96267.96279

Digital Library

[55]

Hoang Lam Nguyen and Lars Grunske. 2022. BeDivFuzz: Integrating Behavioral Diversity into Generator-Based Fuzzing. In Proceedings of the 44th International Conference on Software Engineering (ICSE ’22). Association for Computing Machinery, New York, NY, USA. 249–261. isbn:9781450392211 https://doi.org/10.1145/3510003.3510182

Digital Library

[56]

Hoang Lam Nguyen, Nebras Nassar, Timo Kehrer, and Lars Grunske. 2020. MoFuzz: A fuzzer suite for testing model-driven software engineering tools. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1103–1115. https://doi.org/10.1145/3324884.3416668

Digital Library

[57]

A. Jefferson Offutt, Ammei Lee, Gregg Rothermel, Roland H. Untch, and Christian Zapf. 1996. An Experimental Determination of Sufficient Mutant Operators. ACM Trans. Softw. Eng. Methodol., 5, 2 (1996), apr, 99–118. issn:1049-331X https://doi.org/10.1145/227607.227610

Digital Library

[58]

Rohan Padhye, Caroline Lemieux, and Koushik Sen. 2019. JQF: Coverage-guided Property-based Testing in Java. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’19). 398–401. isbn:978-1-4503-6224-5 https://doi.org/10.1145/3293882.3339002

Digital Library

[59]

Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, and Yves Le Traon. 2019. Semantic Fuzzing with Zest. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2019). ACM, 329–340. isbn:978-1-4503-6224-5 https://doi.org/10.1145/3293882.3330576

Digital Library

[60]

Rohan Padhye, Caroline Lemieux, Koushik Sen, Laurent Simon, and Hayawardh Vijayakumar. 2019. FuzzFactory: domain-specific fuzzing with waypoints. Proceedings of the ACM on Programming Languages, 3, OOPSLA (2019), Article 174, 29 pages. https://doi.org/10.1145/3360600

Digital Library

[61]

Rohan Padhye, Koushik Sen, and Paul N. Hilfinger. 2019. ChocoPy: A Programming Language for Compilers Courses. In Proceedings of the 2019 ACM SIGPLAN Symposium on SPLASH-E (SPLASH-E 2019). Association for Computing Machinery, 41–45. isbn:9781450369893 https://doi.org/10.1145/3358711.3361627

Digital Library

[62]

Mike Papadakis, Christopher Henard, Mark Harman, Yue Jia, and Yves Le Traon. 2016. Threats to the validity of mutation-based test assessment. In Proceedings of the 25th International Symposium on Software Testing and Analysis. 354–365. https://doi.org/10.1145/2931037.2931040

Digital Library

[63]

Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman. 2019. Mutation testing advances: an analysis and survey. In Advances in Computers. 112, Elsevier, 275–378. https://doi.org/10.1016/bs.adcom.2018.03.015

[64]

Mike Papadakis, Donghwan Shin, Shin Yoo, and Doo-Hwan Bae. 2018. Are mutation scores correlated with real fault detection? a large scale empirical study on the relationship between mutants and real faults. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). 537–548. https://doi.org/10.1145/3180155.3180183

Digital Library

[65]

Van-Thuan Pham, Marcel Böhme, Andrew Edward Santosa, Alexandru Razvan Caciulescu, and Abhik Roychoudhury. 2019. Smart greybox fuzzing. IEEE Transactions on Software Engineering, https://doi.org/10.1109/TSE.2019.2941681

[66]

Alessandro Viola Pizzoleto, Fabiano Cutigi Ferrari, Jeff Offutt, Leo Fernandes, and Márcio Ribeiro. 2019. A systematic literature review of techniques and metrics to reduce the cost of mutation testing. Journal of Systems and Software, 157 (2019), 110388. https://doi.org/10.1016/j.jss.2019.07.100

[67]

Ruixiang Qian, Quanjun Zhang, Chunrong Fang, and Lihua Guo. 2022. Investigating Coverage Guided Fuzzing with Mutation Testing. In Proceedings of the 13th Asia-Pacific Symposium on Internetware (Internetware ’22). Association for Computing Machinery, 272–281. isbn:9781450397803 https://doi.org/10.1145/3545258.3545285

Digital Library

[68]

Alexandre Rebert, Sang Kil Cha, Thanassis Avgerinos, Jonathan Foote, David Warren, Gustavo Grieco, and David Brumley. 2014. Optimizing seed selection for fuzzing. In 23rd USENIX Security Symposium (USENIX Security 14). 861–875. https://doi.org/10.5555/2671225.2671280

Digital Library

[69]

Sameer Reddy, Caroline Lemieux, Rohan Padhye, and Koushik Sen. 2020. Quickly generating diverse valid test inputs with reinforcement learning. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). 1410–1421. https://doi.org/10.1145/3377811.3380399

Digital Library

[70]

David Schuler and Andreas Zeller. 2009. Javalanche: Efficient Mutation Testing for Java. In Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on The Foundations of Software Engineering (ESEC/FSE ’09). Association for Computing Machinery, 297–298. isbn:9781605580012 https://doi.org/10.1145/1595696.1595750

Digital Library

[71]

SQLite Authors. 2019. How SQLite is Tested. https://www.sqlite.org/testing.html#the_fuzzcheck_test_harness Retrieved August 31, 2022

[72]

The OpenSSL Project. 2016. Run the fuzzing corpora as tests. https://github.com/openssl/openssl/commit/90d28f05 Retrieved August 31, 2022

[73]

Macario Polo Usaola and Pedro Reales Mateo. 2010. Mutation Testing Cost Reduction Techniques: A Survey. IEEE Software, 27, 3 (2010), 80–86. https://doi.org/10.1109/MS.2010.79

Digital Library

[74]

Vasudev Vikram, Isabella Laybourn, Ao Li, Nicole Nair, Kelton OBrien, Rafaello Sanna, and Rohan Padhye. 2023. Mu2: Guiding Greybox Fuzzing with Mutation Testing (Artifact). https://doi.org/10.5281/zenodo.8006662

Digital Library

[75]

Vasudev Vikram, Rohan Padhye, and Koushik Sen. 2021. Growing A Test Corpus with Bonsai Fuzzing. In 43rd IEEE/ACM International Conference on Software Engineering, ICSE 2021, Madrid, Spain, 22-30 May 2021. IEEE, 723–735. https://doi.org/10.1109/ICSE43902.2021.00072

Digital Library

[76]

Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2019. Superion: Grammar-Aware Greybox Fuzzing. In 41st International Conference on Software Engineering (ICSE ’19). https://doi.org/10.1109/ICSE.2019.00081

Digital Library

[77]

Pengfei Wang, Xu Zhou, Kai Lu, Tai Yue, and Yingying Liu. 2022. SoK: The progress, challenges, and perspectives of directed greybox fuzzing. arXiv preprint, https://doi.org/10.48550/arXiv.2005.11907

[78]

Cheng Wen. 2022. Recent Papers Related To Fuzzing. https://wcventure.github.io/FuzzingPaper/ Retrieved March 16, 2022

[79]

Elaine J Weyuker. 1986. Axiomatizing software test data adequacy. IEEE transactions on software engineering, 1128–1138.

[80]

Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and Understanding Bugs in C Compilers. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation (PLDI ’11). Association for Computing Machinery, 283–294. isbn:9781450306638 https://doi.org/10.1145/1993498.1993532

Digital Library

[81]

Michal Zalewski. 2014. American Fuzzy Lop. https://lcamtuf.coredump.cx/afl/ Accessed February 11, 2022

[82]

Jie Zhang, Ziyi Wang, Lingming Zhang, Dan Hao, Lei Zang, Shiyang Cheng, and Lu Zhang. 2016. Predictive Mutation Testing. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). Association for Computing Machinery, New York, NY, USA. 342–353. isbn:9781450343909 https://doi.org/10.1145/2931037.2931038

Digital Library

[83]

Qian Zhang, Jiyuan Wang, Muhammad Ali Gulzar, Rohan Padhye, and Miryung Kim. 2020. Bigfuzz: Efficient fuzz testing for data analytics using framework abstraction. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 722–733. https://doi.org/10.1145/3324884.3416641

Digital Library

[84]

Xiaogang Zhu and Marcel Böhme. 2021. Regression Greybox Fuzzing. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS ’21). Association for Computing Machinery, 2169–2182. isbn:9781450384544 https://doi.org/10.1145/3460120.3484596

Digital Library

Cited By

Zhou ZYang YWu SHuang YChen BPeng XFilkov VRay BZhou M(2024)Magneto: A Step-Wise Approach to Exploit Vulnerabilities in Dependent Libraries via LLM-Empowered Directed FuzzingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695531(1633-1644)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695531
Padhye REdwards JTaeumel M(2024)Software Engineering Methods for AI-Driven Deductive Legal ReasoningProceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3689492.3690050(85-95)Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3689492.3690050
Li AHuang MLemieux CPadhye RBöhme MNoller YSzekeres L(2024)The Havoc Paradox in Generator-Based Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685529(3-12)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685529
Show More Cited By

Index Terms

Guiding Greybox Fuzzing with Mutation Testing
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Fuzzing for CPS Mutation Testing
ASE '23: Proceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering

Mutation testing can help reduce the risks of releasing faulty software. For such reason, it is a desired practice for the development of embedded software running in safety-critical cyber-physical systems (CPS). Unfortunately, state-of-the-art test data ...
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...
Mitigating the effects of flaky tests on mutation testing
ISSTA 2019: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

Mutation testing is widely used in research as a metric for evaluating the quality of test suites. Mutation testing runs the test suite on generated mutants (variants of the code under test), where a test suite kills a mutant if any of the tests fail ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 2023

1554 pages

ISBN:9798400702211

DOI:10.1145/3597926

General Chair:
René Just
University of Washington, USA
,
Program Chair:
Gordon Fraser
University of Passau, Germany

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution 4.0 International License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

ISSTA '23

Sponsor:

SIGSOFT

ISSTA '23: 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

July 17 - 21, 2023

WA, Seattle, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,583
Total Downloads

Downloads (Last 12 months)980
Downloads (Last 6 weeks)113

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou ZYang YWu SHuang YChen BPeng XFilkov VRay BZhou M(2024)Magneto: A Step-Wise Approach to Exploit Vulnerabilities in Dependent Libraries via LLM-Empowered Directed FuzzingProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695531(1633-1644)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695531
Padhye REdwards JTaeumel M(2024)Software Engineering Methods for AI-Driven Deductive Legal ReasoningProceedings of the 2024 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3689492.3690050(85-95)Online publication date: 17-Oct-2024
https://dl.acm.org/doi/10.1145/3689492.3690050
Li AHuang MLemieux CPadhye RBöhme MNoller YSzekeres L(2024)The Havoc Paradox in Generator-Based Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685529(3-12)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678722.3685529
Qian RZhang QFang CYang DLi SLi BChen Z(2024)DiPri: Distance-based Seed Prioritization for Greybox FuzzingACM Transactions on Software Engineering and Methodology10.1145/3654440Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3654440
Yu XLiu LHu XKeung JXia XLo DChristakis MPradel M(2024)Practitioners’ Expectations on Automated Test GenerationProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680386(1618-1630)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680386
Ouyang YYang JZhang LChristakis MPradel M(2024)Benchmarking Automated Program Repair: An Extensive Study on Both Real-World and Artificial BugsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652140(440-452)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652140
Fan ZYe GHu TTang Z(2024)History-driven Compiler Fuzzing via Assembling and Scheduling Bug-triggering Code Segments2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE62328.2024.00040(331-342)Online publication date: 28-Oct-2024
https://doi.org/10.1109/ISSRE62328.2024.00040
Han QWang LZhang HShi LWang D(2024)Ethchecker: a context-guided fuzzing for smart contractsThe Journal of Supercomputing10.1007/s11227-024-05954-980:10(13949-13975)Online publication date: 13-Mar-2024
https://dl.acm.org/doi/10.1007/s11227-024-05954-9
Zuo XSong YMao RQin ZSun X(2023)MODGF: Multi-Objective Optimization Approach for Directed Greybox Fuzzing2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys)10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00024(106-113)Online publication date: 17-Dec-2023
https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00024
Lee JViganò ECornejo OPastore FBriand LBissyandé TKlein JBird CSarro F(2023)Fuzzing for CPS Mutation TestingProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE56229.2023.00079(1377-1389)Online publication date: 11-Nov-2023
https://dl.acm.org/doi/10.1109/ASE56229.2023.00079

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents