[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3634737.3657003acmconferencesArticle/Chapter ViewAbstractPublication Pagesasia-ccsConference Proceedingsconference-collections
research-article
Open access

Cross-Language Differential Testing of JSON Parsers

Published: 01 July 2024 Publication History

Abstract

JSON is a widely used format for representing data on the Internet. Unfortunately, the format is imprecisely specified, which poses the risk of confusion and ambiguity when processing sensitive data. While previous work has focused on manual analysis of parsers, an automatic analysis of the interplay of multiple parsers resulting from this imprecision has received little attention so far. In this paper, we address this problem and propose a framework for differential testing of JSON parsers tailored towards discovering semantic discrepancies. To spot these differences automatically, we overcome two challenges: First, we introduce a consensus-based normalization of JSON that enables us to analyze data semantics in absence of a precise specification. Second, we propose a novel mechanism for tracking test coverage across runtime environments, so that confusions between parsers written in C, C++, Rust, Java, and Python can be detected simultaneously. In a comparative analysis of 22 JSON parsers, we uncover various semantic discrepancies, ranging from minor inconsistencies in the representation of numbers and strings to severe confusions in the handling of object keys and values. We illustrate the security impact of these discrepancies in different case studies, echoing recent efforts to enforce a stricter specification for JSON in security applications.

References

[1]
Atheris: A coverage-guided, native python fuzzer. https://github.com/google/atheris. Accessed: 2023-12-08.
[2]
Frida: Dynamic instrumentation toolkit for developers, reverse-engineers, and security researchers. https://github.com/frida/frida. Accessed: 2023-12-08.
[3]
Jazzer: Fuzz Testing for the JVM. https://github.com/CodeIntelligenceTesting/jazzer. Accessed: 2023-12-08.
[4]
funfuzz. https://github.com/MozillaSecurity/funfuzz. Accessed: 2023-12-08.
[5]
libfuzzer - a library for coverage-guided fuzz testing. https://llvm.org/docs/LibFuzzer.html. Accessed: 2023-12-08.
[6]
CVE-2017-12635. Entry on MITRE, 2017.
[7]
IEEE standard for floating-point arithmetic. IEEE Std 754-2019 (Revision of IEEE 754-2008), pages 1--84, 2019.
[8]
CVE-2023-48891. Entry on MITRE, 2023.
[9]
A. Avizienis. The n-version approach to fault-tolerant software. IEEE Transactions on Software Engineering, SE-11(12):1491--1501, 1985.
[10]
L. Bernhard, T. Scharnowski, M. Schloegel, T. Blazytko, and T. Holz. Jit-picking: Differential fuzzing of javascript engines. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 351--364, 2022.
[11]
T. Bray. The I-JSON Message Format. RFC 7493, 2015.
[12]
T. Bray. The JavaScript Object Notation (JSON) Data Interchange Format. RFC 8259, Dec. 2017. URL https://www.rfc-editor.org/info/rfc8259.
[13]
C. Brubaker, S. Jana, B. Ray, S. Khurshid, and V. Shmatikov. Using frankencerts for automated adversarial testing of certificate validation in ssl/tls implementations. In 2014 IEEE Symposium on Security and Privacy, pages 114--129. IEEE, 2014.
[14]
C. Chen, P. Ren, Z. Duan, C. Tian, X. Lu, and B. Yu. Sbdt: Search-based differential testing of certificate parsers in ssl/tls implementations. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 967--979, 2023.
[15]
Y. Chen and Z. Su. Guided differential testing of certificate validation in ssl/tls implementations. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, pages 793--804, 2015.
[16]
Y. Chen, T. Su, C. Sun, Z. Su, and J. Zhao. Coverage-directed differential testing of jvm implementations. In proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 85--99, 2016.
[17]
Y. Chen, T. Su, and Z. Su. Deep differential testing of jvm implementations. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pages 1257--1268. IEEE, 2019.
[18]
D. Hardt. The OAuth 2.0 Authorization Framework. RFC 6749, 2012.
[19]
N. Harrand, T. Durieux, D. Broman, and B. Baudry. The behavioral diversity of java json libraries. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), pages 412--422. IEEE, 2021.
[20]
B. Jabiyev, S. Sprecher, K. Onarlioglu, and E. Kirda. T-reqs: Http request smuggling with differential fuzzing. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pages 1805--1820, 2021.
[21]
M. B. Jones, J. Bradley, and N. Sakimura. JSON Web Token (JWT). RFC 7519, 2015.
[22]
W. Li, J. Ruan, G. Yi, L. Cheng, X. Luo, and H. Cai. Polyfuzz: Holistic greybox fuzzing of multi-language systems. In 32nd USENIX Security Symposium (USENIX Security 23), 2023.
[23]
W. Li, H. Yang, X. Luo, L. Cheng, and H. Cai. Pyrtfuzz: Detecting bugs in python runtimes via two-level collaborative fuzzing. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 1645--1659, 2023.
[24]
W. M. McKeeman. Differential testing for software. Digital Technical Journal, 10 (1):100--107, 1998.
[25]
J. Miller. An exploration of json interoperability vulnerabilities. https://bishopfox.com/blog/json-interoperability-vulnerabilities. Accessed: 2023-10-05.
[26]
E. Nagai, H. Awazu, N. Ishiura, and N. Takeda. Random testing of c compilers targeting arithmetic optimization. In Workshop on Synthesis And System Integration of Mixed Information Technologies (SASIMI 2012), pages 48--53, 2012.
[27]
E. Nagai, A. Hashimoto, and N. Ishiura. Reinforcing random testing of arithmetic optimization of c compilers by scaling up size and number of expressions. IPSJ Transactions on System LSI Design Methodology, 7:91--100, 2014.
[28]
J. Park, S. An, D. Youn, G. Kim, and S. Ryu. Jest: N+ 1-version differential testing of both javascript engines and specification. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), pages 13--24. IEEE, 2021.
[29]
T. Petsios, A. Tang, S. Stolfo, A. D. Keromytis, and S. Jana. Nezha: Efficient domain-independent differential testing. In 2017 IEEE Symposium on security and privacy (SP), pages 615--632. IEEE, 2017.
[30]
V. J. Reddi, A. Settle, D. A. Connors, and R. S. Cohn. Pin: a binary instrumentation tool for computer architecture research and education. In Proceedings of the 2004 workshop on Computer architecture education: held in conjunction with the 31st International Symposium on Computer Architecture, pages 22--es, 2004.
[31]
A. Rundgren, B. Jordan, and S. Erdtman. JSON Canonicalization Scheme (JCS). RFC 8785, June 2020.
[32]
N. Seriot. Parsing json is a minefield. https://seriot.ch/projects/parsing_json.html. Accessed: 2023-10-05.
[33]
M. Sharma, P. Yu, and A. F. Donaldson. Rustsmith: Random differential compiler testing for rust. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1483--1486, 2023.
[34]
M. Wu, Y. Ouyang, M. Lu, J. Chen, Y. Zhao, H. Cui, Y. Guo, and Y. Zhang. Sjfuzz: Seed & mutator scheduling for jvm fuzzing. In 2023 The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), 2023.
[35]
X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in c compilers. In Proceedings of the 32nd ACM SIGPLAN conference on Programming language design and implementation, pages 283--294, 2011.

Index Terms

  1. Cross-Language Differential Testing of JSON Parsers

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ASIA CCS '24: Proceedings of the 19th ACM Asia Conference on Computer and Communications Security
    July 2024
    1987 pages
    ISBN:9798400704826
    DOI:10.1145/3634737
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 July 2024

    Check for updates

    Author Tags

    1. cross-language
    2. differential testing
    3. JSON specification

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ASIA CCS '24
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 418 of 2,322 submissions, 18%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 282
      Total Downloads
    • Downloads (Last 12 months)282
    • Downloads (Last 6 weeks)84
    Reflects downloads up to 24 Dec 2024

    Other Metrics

    Citations

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media