[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3510003.3510628acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article
Open access

CONFETTI: amplifying concolic guidance for fuzzers

Published: 05 July 2022 Publication History

Abstract

Fuzz testing (fuzzing) allows developers to detect bugs and vulnerabilities in code by automatically generating defect-revealing inputs. Most fuzzers operate by generating inputs for applications and mutating the bytes of those inputs, guiding the fuzzing process with branch coverage feedback via instrumentation. Whitebox guidance (e.g., taint tracking or concolic execution) is sometimes integrated with coverage-guided fuzzing to help cover tricky-to-reach branches that are guarded by complex conditions (so-called "magic values"). This integration typically takes the form of a targeted input mutation, e.g., placing particular byte values at a specific offset of some input in order to cover a branch. However, these dynamic analysis techniques are not perfect in practice, which can result in the loss of important relationships between input bytes and branch predicates, thus reducing the effective power of the technique. We introduce a new, surprisingly simple, but effective technique, global hinting, which allows the fuzzer to insert these interesting bytes not only at a targeted position, but in any position of any input. We implemented this idea in Java, creating Confetti, which uses both targeted and global hints for fuzzing. In an empirical comparison with two baseline approaches, a state-of-the-art greybox Java fuzzer and a version of Confetti without global hinting, we found that Confetti covers more branches and finds 15 previously unreported bugs, including 9 that neither baseline could find. By conducting a post-mortem analysis of Confetti's execution, we determined that global hinting was at least as effective at revealing new coverage as traditional, targeted hinting.

References

[1]
2019. Google Closure Issue Tracker Issue Number 3375. https://github.com/google/closure-compiler/issues/3375.
[2]
2019. Google Closure Issue Tracker Issue Number 3380. https://github.com/google/closure-compiler/issues/3380.
[3]
2019. Google Closure Issue Tracker Issue Number 3455. https://github.com/google/closure-compiler/issues/3455.
[4]
2020. Google Closure Issue Tracker Issue Number 3591. https://github.com/google/closure-compiler/issues/3591.
[5]
2020. Google Closure Issue Tracker Issue Number 3592. https://github.com/google/closure-compiler/issues/3592.
[6]
2021. Apache Ant. https://ant.apache.org/.
[7]
2021. Apache BCEL Issue Tracker Issue Number 357. https://issues.apache.org/jira/projects/BCEL/issues/BCEL-357.
[8]
2021. Apache BCEL Issue Tracker Issue Number 358. https://issues.apache.org/jira/projects/BCEL/issues/BCEL-358.
[9]
2021. Apache BCEL Issue Tracker Issue Number 359. https://issues.apache.org/jira/projects/BCEL/issues/BCEL-359.
[10]
2021. Apache Commons Byte Code Engineering Library. http://commons.apache.org/proper/commons-bcel/.
[11]
2021. Apache Maven. http://maven.apache.org/.
[12]
2021. Google Closure. https://developers.google.com/closure/compiler.
[13]
2021. Google Closure Issue Tracker Issue Number 3857. https://github.com/google/closure-compiler/issues/3857.
[14]
2021. Google Closure Issue Tracker Issue Number 3858. https://github.com/google/closure-compiler/issues/3858.
[15]
2021. Google Closure Issue Tracker Issue Number 3859. https://github.com/google/closure-compiler/issues/3859.
[16]
2021. Google Closure Issue Tracker Issue Number 3860. https://github.com/google/closure-compiler/issues/3860.
[17]
2021. Google Closure Issue Tracker Issue Number 3861. https://github.com/google/closure-compiler/issues/3861.
[18]
2021. Google Closure Issue Tracker Issue Number 3862. https://github.com/google/closure-compiler/issues/3862.
[19]
2021. Mozilla Rhino. https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino.
[20]
AFL Contributors. 2021. AFL Dictionaries. https://github.com/mirrorer/afl/blob/master/dictionaries/README.dictionaries.
[21]
Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated Concolic Testing of Smartphone Apps (FSE '12). Association for Computing Machinery, New York, NY, USA, Article Article 59, 11 pages.
[22]
Saswat Anand, Corina S. Păsăreanu, and Willem Visser. 2007. JPF-SE: A Symbolic Execution Extension to Java PathFinder (TACAS'07). Springer-Verlag, Berlin, Heidelberg, 134--138.
[23]
Daniel Baier, Dirk Beyer, and Karlheinz Friedberger. 2021. JavaSMT 3: Interacting with SMT Solvers in Java. In Computer Aided Verification, Alexandra Silva and K. Rustan M. Leino (Eds.). Springer International Publishing, Cham, 195--208.
[24]
Tao Bao, Yunhui Zheng, Zhiqiang Lin, Xiangyu Zhang, and Dongyan Xu. 2010. Strict Control Dependence and Its Effect on Dynamic Information Flow Analyses (ISSTA '10). ACM, New York, NY, USA, 13--24.
[25]
Jonathan Bell. 2022. Faster, collision-free coverage instrumentation, Pull Request #171 on rohanpadhye/JQF. https://github.com/rohanpadhye/JQF/pull/171.
[26]
Jonathan Bell. 2022. Increase maximum depth of XML files generated for Maven, Pull Request #165 on rohanpadhye/JQF. https://github.com/rohanpadhye/JQF/pull/165.
[27]
Jonathan Bell and Gail Kaiser. 2014. Phosphor - GitHub. https://github.com/gmu-swe/phosphor.
[28]
Jonathan Bell and Gail Kaiser. 2014. Phosphor: Illuminating Dynamic Data Flow in Commodity JVMs (OOPSLA '14). ACM, New York, NY, USA, 83--101.
[29]
Jonathan Bell and Luís Pina. 2018. CROCHET: Checkpoint and Rollback via Lightweight Heap Traversal on Stock JVMs (ECOOP '18). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
[30]
Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed Greybox Fuzzing (CCS '17). ACM, New York, NY, USA, 2329--2344.
[31]
Ella Bounimova, Patrice Godefroid, and David Molnar. 2013. Billions and Billions of Constraints: Whitebox Fuzz Testing in Production (ICSE '13). IEEE Press, 122--131.
[32]
Cristian Cadar, Daniel Dunbar, and Dawson Engler. 2008. KLEE: Unassisted and Automatic Generation of High-coverage Tests for Complex Systems Programs (OSDI'08). USENIX Association, Berkeley, CA, USA, 209--224. http://dl.acm.org/citation.cfm?id=1855741.1855756
[33]
P. Chen and H. Chen. 2018. Angora: Efficient Fuzzing by Principled Search. In 2018 IEEE Symposium on Security and Privacy (SP). 711--725.
[34]
Peng Chen, Jianzhong Liu, and Hao Chen. 2019. Matryoshka: Fuzzing Deeply Nested Branches (CCS '19). Association for Computing Machinery, New York, NY, USA, 499--513.
[35]
Jaeseung Choi, Joonun Jang, Choongwoo Han, and Sang Kil Cha. 2019. Grey-Box Concolic Testing on Binary Code (ICSE '19). IEEE Press, 736--747.
[36]
Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337--340.
[37]
Stephen Dolan. 2017. Property fuzzing for OCaml. https://github.com/stedolan/crowbar.
[38]
Andrea Fioraldi, Dominik Maier, Heiko Eibfeldt, and Marc Heuse. 2020. AFL++: Combining incremental steps of fuzzing research. In 14th USENIX Workshop on Offensive Technologies (WOOT 20).
[39]
Shuitao Gan, Chao Zhang, Peng Chen, Bodong Zhao, Xiaojun Qin, Dong Wu, and Zuoning Chen. 2020. GREYONE: Data Flow Sensitive Fuzzing. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 2577--2594. https://www.usenix.org/conference/usenixsecurity20/presentation/gan
[40]
Vijay Ganesh, Tim Leek, and Martin Rinard. 2009. Taint-Based Directed Whitebox Fuzzing. In Proceedings of the 31st International Conference on Software Engineering (ICSE '09). IEEE Computer Society, USA, 474--484.
[41]
Patrice Godefroid, Adam Kiezun, and Michael Levin. 2008. Grammar-based Whitebox Fuzzing. Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) 43, 206--215.
[42]
Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed Automated Random Testing (PLDI '05). Association for Computing Machinery, New York, NY, USA, 213--223.
[43]
Patrice Godefroid, Hila Peleg, and Rishabh Singh. 2017. Learn&Fuzz: Machine Learning for Input Fuzzing (ASE 2017). IEEE Press, 50--59.
[44]
Google. 2022. honggfuzz. https://honggfuzz.dev.
[45]
Istvan Haller, Asia Slowinska, Matthias Neugschwandtner, and Herbert Bos. 2013. Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations. In 22nd USENIX Security Symposium (USENIX Security 13). USENIX Association, Washington, D.C., 49--64. https://www.usenix.org/conference/usenixsecurity13/technical-sessions/papers/haller
[46]
Paul R. Holser. 2010. junit-quickcheck: Property-based testing, JUnit-style. https://github.com/pholser/junit-quickcheck.
[47]
Katherine Hough and Jonathan Bell. 2021. A Practical Approach for Dynamic Taint Tracking with Control-Flow Relationships. ACM Trans. Softw. Eng. Methodol. 31, 2, Article 26 (dec 2021), 43 pages.
[48]
Karthick Jayaraman, David Harvison, and Vijay Ganesh. 2009. jFuzz: A Concolic Whitebox Fuzzer for Java. In Proceedings of the 1st NASA Formal Methods Symposium (NFM).
[49]
Min Gyung Kang, Stephen McCamant, Pongsin Poosankam, and Dawn Xiaodong Song. 2011. DTA++: Dynamic Taint Analysis with Targeted Control-Flow Propagation. In NDSS.
[50]
George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating Fuzz Testing (CCS '18). Association for Computing Machinery, New York, NY, USA, 2123--2138.
[51]
James Kukucka, Luís Pina, Paul Ammann, and Jonathan Bell. 2019. CONFETTI: Amplifying Concolic Guidance for Fuzzers. https://github.com/neuse/CONFETTI.
[52]
James Kukucka, Luís Pina, Paul Ammann, and Jonathan Bell. 2022. CONFETTI: Amplifying Concolic Guidance for Fuzzers.
[53]
Leonidas Lampropoulos, Michael Hicks, and Benjamin C. Pierce. 2019. Coverage Guided, Property Based Testing. Proc. ACM Program. Lang. 3, OOPSLA, Article 181 (Oct. 2019), 29 pages.
[54]
libFuzzer Contributors. 2021. libFuzzer Tutorial. https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md#dictionaries.
[55]
LLVM Project. 2019. libFuzzer - a library for coverage-guided fuzz testing. https://llvm.org/docs/LibFuzzer.html.
[56]
Kasper Luckow, Marko Dimjašević, Dimitra Giannakopoulou, Falk Howar, Malte Isberner, Temesghen Kahsai, Zvonimir Rakamarić, and Vishwanath Raman. 2016. JDart: A Dynamic Symbolic Analysis Framework. In Tools and Algorithms for the Construction and Analysis of Systems. Springer Berlin Heidelberg, 442--459.
[57]
Rupak Majumdar and Koushik Sen. 2007. Hybrid Concolic Testing. In Proceedings of the 29th International Conference on Software Engineering (ICSE '07). IEEE Computer Society, USA, 416--426.
[58]
Björn Mathis, Rahul Gopinath, and Andreas Zeller. 2020. Learning Input Tokens for Effective Fuzzing (ISSTA 2020). Association for Computing Machinery, New York, NY, USA, 2737.
[59]
Mountainminds GmbH & Co. KG and Contributors. 2021. JaCoCo Java Code Coverage Library. http://www.eclemma.org/jacoco/.
[60]
Rickard Nilsson. 2019. ScalaCheck: Property-based testing for Scala. https://www.scalacheck.org.
[61]
Yannic Noller, Rody Kersten, and Corina S. Păsăreanu. 2018. Badger: Complexity Analysis with Fuzzing and Symbolic Execution (ISSTA 2018). Association for Computing Machinery, New York, NY, USA, 322--332.
[62]
Yannic Noller, Corina S. Păsăreanu, Marcel Böhme, Youcheng Sun, Hoang Lam Nguyen, and Lars Grunske. 2020. HyDiff: Hybrid Differential Software Analysis (ICSE '20). Association for Computing Machinery, New York, NY, USA, 1273--1285.
[63]
Saahil Ognawala, Thomas Hutzelmann, Eirini Psallida, and Alexander Pretschner. 2018. Improving Function Coverage with Munch: A Hybrid Fuzzing and Directed Symbolic Execution Approach (SAC '18). Association for Computing Machinery, New York, NY, USA, 1475--1482.
[64]
Rohan Padhye, Caroline Lemieux, and Koushik Sen. 2019. JQF + Zest: Coverage-guided semantic fuzzing for Java. https://github.com/rohanpadhye/JQF.
[65]
Rohan Padhye, Caroline Lemieux, and Koushik Sen. 2019. Jqf: Coverage-guided property-based testing in java. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 398--401.
[66]
Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, and Yves Le Traon. 2019. Semantic fuzzing with zest. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 329--340.
[67]
Jibesh Patra and Michael Pradel. 2016. Learning to fuzz: Application-independent fuzz testing with probabilistic, generative models of input data. TU Darmstadt, Department of Computer Science, Tech. Rep. TUD-CS-2016-14664 (2016).
[68]
Sebastian Poeplau and Aurélien Francillon. 2020. Symbolic execution with SymCC: Don't interpret, compile!. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, 181--198. https://www.usenix.org/conference/usenixsecurity20/presentation/poeplau
[69]
Sanjay Rawat, Vivek Jain, Ashish Kumar, Lucian Cojocar, Cristiano Giuffrida, and Herbert Bos. 2017. VUzzer: Application-aware Evolutionary Fuzzing. In NDSS. https://www.vusec.net/download/?t=papers/vuzzer_ndss17.pdf
[70]
Koushik Sen and Gul Agha. 2006. CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools. In CAV, Thomas Ball and Robert B. Jones (Eds.). 419--423.
[71]
Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A Concolic Unit Testing Engine for C (ESEC/FSE-13). ACM, New York, NY, USA, 263--272.
[72]
Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting Fuzzing Through Selective Symbolic Execution. In 23rd Annual Network and Distributed System Security Symposium, NDSS 2016, San Diego, California, USA, February 21--24, 2016. The Internet Society. http://wp.internetsociety.org/ndss/wp-content/uploads/sites/25/2017/09/driller-augmenting-fuzzing-through-selective-symbolic-execution.pdf
[73]
H. Tanno, X. Zhang, T. Hoshino, and K. Sen. 2015. TesMa and CATG: Automated Test Generation Tools for Models of Enterprise Applications. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 2. 717--720.
[74]
Willem Visser and Jaco Geldenhuys. 2020. COASTAL: Combining Concolic and Fuzzing for Java (Competition Contribution). In Tools and Algorithms for the Construction and Analysis of Systems, Armin Biere and David Parker (Eds.). Springer International Publishing, Cham, 373--377.
[75]
Willem Visser, Jaco Geldenhuys, and Matthew B. Dwyer. 2012. Green: Reducing, Reusing and Recycling Constraints in Program Analysis (FSE '12). Association for Computing Machinery, New York, NY, USA, Article 58, 11 pages.
[76]
Willem Visser, Klaus Havelund, Guillaume Brat, Seungjoon Park, and Flavio Lerda. 2003. Model Checking Programs. Automated Software Engg. 10, 2 (April 2003), 203--232.
[77]
J. Wang, B. Chen, L. Wei, and Y. Liu. 2017. Skyfire: Data-Driven Seed Generation for Fuzzing. In 2017 IEEE Symposium on Security and Privacy (SP). 579--594.
[78]
Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2019. Superion: Grammar-Aware Greybox Fuzzing (ICSE '19). IEEE Press, 724--735.
[79]
Mingzhe Wang, Jie Liang, Yuanliang Chen, Yu Jiang, Xun Jiao, Han Liu, Xibin Zhao, and Jiaguang Sun. 2018. SAFL: Increasing and Accelerating Testing Coverage with Symbolic Execution and Guided Fuzzing (ICSE '18). Association for Computing Machinery, New York, NY, USA, 61--64.
[80]
Tielei Wang, Tao Wei, Guofei Gu, and Wei Zou. 2011. Checksum-Aware Fuzzing Combined with Dynamic Taint Analysis and Symbolic Execution. ACM Trans. Inf. Syst. Secur. 14, 2, Article Article 15 (Sept. 2011), 28 pages.
[81]
Michal Zalewski. 2019. American Fuzzy Lop. http://lcamtuf.coredump.cx/afl/technical_details.txt.
[82]
Lei Zhao, Yue Duan, Heng Yin, and Jifeng Xuan. 2019. Send Hardest Problems My Way: Probabilistic Path Prioritization for Hybrid Fuzzing. In Proceedings 2019 Network and Distributed System Security Symposium. Internet Society, San Diego, CA.

Cited By

View all
  • (2024)The Havoc Paradox in Generator-Based Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685529(3-12)Online publication date: 13-Sep-2024
  • (2024)An Empirical Examination of Fuzzer Mutator PerformanceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680387(1631-1642)Online publication date: 11-Sep-2024
  • (2024)Generator-based Fuzzing with Input FeaturesProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643925(13-20)Online publication date: 14-Apr-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
ICSE '22: Proceedings of the 44th International Conference on Software Engineering
May 2022
2508 pages
ISBN:9781450392211
DOI:10.1145/3510003
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2022

Permissions

Request permissions for this article.

Check for updates

Badges

Qualifiers

  • Research-article

Funding Sources

Conference

ICSE '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)244
  • Downloads (Last 6 weeks)44
Reflects downloads up to 13 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)The Havoc Paradox in Generator-Based Fuzzing (Registered Report)Proceedings of the 3rd ACM International Fuzzing Workshop10.1145/3678722.3685529(3-12)Online publication date: 13-Sep-2024
  • (2024)An Empirical Examination of Fuzzer Mutator PerformanceProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680387(1631-1642)Online publication date: 11-Sep-2024
  • (2024)Generator-based Fuzzing with Input FeaturesProceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing10.1145/3643659.3643925(13-20)Online publication date: 14-Apr-2024
  • (2024)Crossover in Parametric FuzzingProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639160(1-12)Online publication date: 20-May-2024
  • (2024)Concrete Constraint Guided Symbolic ExecutionProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639078(1-12)Online publication date: 20-May-2024
  • (2024)SoK: Prudent Evaluation Practices for Fuzzing2024 IEEE Symposium on Security and Privacy (SP)10.1109/SP54263.2024.00137(1974-1993)Online publication date: 19-May-2024
  • (2023)Demystify the Fuzzing Methods: A Comprehensive SurveyACM Computing Surveys10.1145/362337556:3(1-38)Online publication date: 5-Oct-2023
  • (2023)Enhancing Coverage-Guided Fuzzing via Phantom ProgramProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616294(1037-1049)Online publication date: 30-Nov-2023
  • (2023)DiPri: Distance-Based Seed Prioritization for Greybox Fuzzing (Registered Report)Proceedings of the 2nd International Fuzzing Workshop10.1145/3605157.3605172(21-30)Online publication date: 17-Jul-2023
  • (2023)Guiding Greybox Fuzzing with Mutation TestingProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598107(929-941)Online publication date: 12-Jul-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media