More Web Proxy on the site http://driver.im/

Article

EnFuzz: ensemble fuzzing with seed synchronization among diverse fuzzers

Authors:

Yuanliang Chen,

Zhuo SuAuthors Info & Claims

SEC'19: Proceedings of the 28th USENIX Conference on Security Symposium

Pages 1967 - 1983

Published: 14 August 2019 Publication History

Abstract

Fuzzing is widely used for vulnerability detection. There are various kinds of fuzzers with different fuzzing strategies, and most of them perform well on their targets. However, in industrial practice, it is found that the performance of those well-designed fuzzing strategies is challenged by the complexity and diversity of real-world applications. In this paper, we systematically study an ensemble fuzzing approach. First, we define the diversity of base fuzzers in three heuristics: diversity of coverage information granularity, diversity of input generation strategy and diversity of seed selection and mutation strategy. Based on those heuristics, we choose several of the most recent base fuzzers that are as diverse as possible, and propose a globally asynchronous and locally synchronous (GALS) based seed synchronization mechanism to seamlessly ensemble those base fuzzers and obtain better performance. For evaluation, we implement EnFuzz based on several widely used fuzzers such as QSYM and FairFuzz, and then we test them on LAVA-M and Google's fuzzing-test-suite, which consists of 24 widely used real-world applications. This experiment indicates that, under the same constraints for resources, these base fuzzers perform differently on different applications, while EnFuzz always outperforms other fuzzers in terms of path coverage, branch coverage and bug discovery. Furthermore, EnFuzz found 60 new vulnerabilities in several well-fuzzed projects such as libpng and libjpeg, and 44 new CVEs were assigned.

References

[1]

Fuzzer automation with spike. http://resources.infosecinstitute.com/fuzzer-automation-with-spike/. [Online; accessed 12-February-2018].

[2]

Cert bff - basic fuzzing framework. https://vuls.cert.org/confluence/display/tools/CERT+BFF+-+Basic+Fuzzing+Framework, 2012. [Online; accessed 10-April-2018].

[3]

Afl in parallel mode. https://github.com/mcarpenter/afl/blob/master/docs/parallel_fuzzing.txt, 2016. [Online; accessed 10-April-2019].

[4]

Continuous fuzzing for open source software. https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html, 2016. [Online; accessed 10-April-2018].

[5]

Google. honggfuzz. https://google.github.io/honggfuzz/, 2016. [Online; accessed 10-April-2018].

[6]

libfuzzer in parallel mode. https://github.com/google/fuzzer-test-suite/blob/master/tutorial/libFuzzerTutorial.md, 2016. [Online; accessed 10-April-2019].

[7]

Technical details for afl. http://lcamtuf.coredump.cx/afl/technical_details.txt, 2016. [Online; accessed 10-April-2019].

[8]

fuzzer-test-suite. https://github.com/google/fuzzer-test-suite, 2017. [Online; accessed 10- April-2018].

[9]

Google security blog. https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html, 2017. [Online; accessed 10-April-2018].

[10]

libfuzzer. https://llvm.org/docs/LibFuzzer.html, 2017. [Online; accessed 10-April-2018].

[11]

Sanitizercoverage in llvm. https://clang.llvm.org/docs/SanitizerCoverage.html, 2017. [Online; accessed 10-April-2018].

[12]

Clusterfuzz document. https://github.com/google/oss-fuzz/blob/master/docs/clusterfuzz.md, 2018. [Online; accessed 2- November-2018].

[13]

Clusterfuzz integration document. https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/HEAD/clusterfuzz.md, 2018. [Online; accessed 2-November-2018].

[14]

BENJAMIN, J. R., AND CORNELL, C. A. Probability, statistics, and decision for civil engineers. Courier Corporation, 2014.

[15]

BÖHME, M., PHAM, V.-T., NGUYEN, M.-D., AND ROYCHOUDHURY, A. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS17) (2017).

Digital Library

[16]

BÖHME, M., PHAM, V.-T., AND ROYCHOUDHURY, A. Coverage-based greybox fuzzing as markov chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (2016), ACM, pp. 1032-1043.

Digital Library

[17]

CHA, S. K., WOO, M., AND BRUMLEY, D. Program-adaptive mutational fuzzing. In Security and Privacy (SP), 2015 IEEE Symposium on (2015), IEEE, pp. 725- 741.

Digital Library

[18]

CHEN, P., AND CHEN, H. Angora: Efficient fuzzing by principled search. arXiv preprint arXiv:1803.01307 (2018).

[19]

DOLAN-GAVITT, B., HULIN, P., KIRDA, E., LEEK, T., MAMBRETTI, A., ROBERTSON, W., ULRICH, F., AND WHELAN, R. Lava: Large-scale automated vulnerability addition. In Security and Privacy (SP), 2016 IEEE Symposium on (2016), IEEE, pp. 110-121.

[20]

EDDINGTON, M. Peach fuzzing platform. Peach Fuzzer (2011), 34.

[21]

GODEFROID, P., KIEZUN, A., AND LEVIN, M. Y. Grammar-based whitebox fuzzing. In ACM Sigplan Notices (2008), vol. 43, ACM, pp. 206-215.

Digital Library

[22]

HELIN, A. Radamsa. https://gitlab.com/akihe/radamsa, 2016.

[23]

HOCEVAR, S. zzuf - multi-purpose fuzzer. http://caca.zoy.org/wiki/zzuf, 2007. [Online; accessed 10-April-2018].

[24]

HOLLER, C., HERZIG, K., AND ZELLER, A. Fuzzing with code fragments. In USENIX Security Symposium (2012), pp. 445-458.

Digital Library

[25]

KLEES, G., RUEF, A., COOPER, B., WEI, S., AND HICKS, M. Evaluating fuzz testing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (2018), ACM, pp. 2123-2138.

Digital Library

[26]

LEMIEUX, C., AND SEN, K. Fairfuzz: Targeting rare branches to rapidly increase greybox fuzz testing coverage. arXiv preprint arXiv:1709.07101 (2017).

[27]

LIANG, J., JIANG, Y., CHEN, Y., WANG, M., ZHOU, C., AND SUN, J. Pafl: extend fuzzing optimizations of single mode to industrial parallel mode. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2018), ACM, pp. 809-814.

Digital Library

[28]

LIANG, J., WANG, M., CHEN, Y., JIANG, Y., AND ZHANG, R. Fuzz testing in practice: Obstacles and solutions. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER) (2018), IEEE, pp. 562-566.

[29]

OGNAWALA, S., HUTZELMANN, T., PSALLIDA, E., AND PRETSCHNER, A. Improving function coverage with munch: a hybrid fuzzing and directed symbolic execution approach. In Proceedings of the 33rd Annual ACM Symposium on Applied Computing (2018), ACM, pp. 1475-1482.

Digital Library

[30]

PETSIOS, T., ZHAO, J., KEROMYTIS, A. D., AND JANA, S. Slowfuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017), ACM, pp. 2155-2168.

Digital Library

[31]

SIRER, E. G., AND BERSHAD, B. N. Using production grammars in software testing. In ACM SIGPLAN Notices (1999), vol. 35, ACM, pp. 1-13.

[32]

STEPHENS, N., GROSEN, J., SALLS, C., DUTCHER, A., WANG, R., CORBETTA, J., SHOSHITAISHVILI, Y., KRUEGEL, C., AND VIGNA, G. Driller: Augmenting fuzzing through selective symbolic execution. In NDSS (2016), vol. 16, pp. 1-16.

[33]

VEGGALAM, S., RAWAT, S., HALLER, I., AND BOS, H. Ifuzzer: An evolutionary interpreter fuzzer using genetic programming. In European Symposium on Research in Computer Security (2016), Springer, pp. 581-601.

[34]

WANG, J., CHEN, B., WEI, L., AND LIU, Y. Skyfire: Data-driven seed generation for fuzzing, 2017.

[35]

WANG, M., LIANG, J., CHEN, Y., JIANG, Y., JIAO, X., LIU, H., ZHAO, X., AND SUN, J. Safl: increasing and accelerating testing coverage with symbolic execution and guided fuzzing. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings (2018), ACM, pp. 61-64.

Digital Library

[36]

XU, W., KASHYAP, S., MIN, C., AND KIM, T. Designing new operating primitives to improve fuzzing performance. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (2017), ACM, pp. 2313-2328.

Digital Library

[37]

YANG, X., CHEN, Y., EIDE, E., AND REGEHR, J. Finding and understanding bugs in c compilers. In ACM SIGPLAN Notices (2011), vol. 46, ACM, pp. 283-294.

Digital Library

[38]

YUN, I., LEE, S., XU, M., JANG, Y., AND KIM, T. fQSYMg: A practical concolic execution engine tailored for hybrid fuzzing. In 27th {USENIX} Security Symposium ({USENIX} Security 18) (2018), pp. 745- 761.

Digital Library

[39]

ZALEWSKI, M. American fuzzy lop. https://github.com/mcarpenter/afl, 2015.

Cited By

Groce ADe Meuter WBaniassad E(2021)Let a thousand flowers bloom: on the uses of diversity in software testingProceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3486607.3486772(136-144)Online publication date: 20-Oct-2021
https://dl.acm.org/doi/10.1145/3486607.3486772

EnFuzz: ensemble fuzzing with seed synchronization among diverse fuzzers
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis

Recommendations

A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...
Spartan: a spectral and entropy-based partial-scan and test point insertion algorithm
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

SEC'19: Proceedings of the 28th USENIX Conference on Security Symposium

August 2019

2002 pages

ISBN:9781939133069

Program Chairs:
Nadia Heninger
University of Pennsylvania
,
Patrick Traynor
University of Florida

Sponsors

Google Inc.
IBMR: IBM Research
Microsoft: Microsoft
Intel: Intel
Facebook: Facebook

Publisher

USENIX Association

United States

Publication History

Published: 14 August 2019

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Groce ADe Meuter WBaniassad E(2021)Let a thousand flowers bloom: on the uses of diversity in software testingProceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software10.1145/3486607.3486772(136-144)Online publication date: 20-Oct-2021
https://dl.acm.org/doi/10.1145/3486607.3486772

View Options

View options

Media

Figures

Other

Tables

View Table of Contents