research-article

Open access

SynTest-solidity: automated test case generation and fuzzing for smart contracts

Authors:

Mitchell Olsthoorn,

Dimitri Stallenberg,

Arie van Deursen,

Annibale PanichellaAuthors Info & Claims

ICSE '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings

Pages 202 - 206

https://doi.org/10.1145/3510454.3516869

Published: 19 October 2022 Publication History

PDF eReader

Abstract

Ethereum is the largest and most prominent smart contract platform. One key property of Ethereum is that once a contract is deployed, it can not be updated anymore. This increases the importance of thoroughly testing the behavior and constraints of the smart contract before deployment. Existing approaches in related work either do not scale or are only focused on finding crashing inputs. In this tool demo, we introduce SynTest-Solidity, an automated test case generation and fuzzing framework for Solidity. SynTest-Solidity implements various metaheuristic search algorithms, including random search (traditional fuzzing) and genetic algorithms (i.e., NSGA-II, MOSA, and DynaMOSA). Finally, we performed a preliminary empirical study to assess the effectiveness of SynTest-Solidity in testing Solidity smart contracts.

References

[1]

Christel Baier and Joost-Pieter Katoen. 2008. Principles of model checking. MIT press.

Digital Library

Google Scholar

[2]

Stefan Driessen, Dario Di Nucci, Geert Monsieur, and Willem-Jan van den Heuvel. 2021. AGSolT: a Tool for Automated Test-Case Generation for Solidity Smart Contracts. arXiv preprint arXiv:2102.08864 (2021).

Google Scholar

[3]

Lu Liu, Lili Wei, Wuqi Zhang, Ming Wen, Yepang Liu, and Shing-Chi Cheung. 2021. Characterizing Transaction-Reverting Statements in Ethereum Smart Contracts. arXiv preprint arXiv:2108.10799 (2021).

Google Scholar

[4]

Tai D Nguyen, Long H Pham, Jun Sun, Yun Lin, and Quang Tran Minh. 2020. sfuzz: An efficient adaptive fuzzer for solidity smart contracts. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 778--788.

Digital Library

Google Scholar

[5]

Annibale Panichella, Fitsum Meshesha Kifetew, and Paolo Tonella. 2015. Reformulating branch coverage as a many-objective optimization problem. In 2015 IEEE 8th international conference on software testing, verification and validation (ICST). IEEE, 1--10.

Crossref

Google Scholar

[6]

Annibale Panichella, Fitsum Mesheha Kifetew, and Paolo Tonella. 2018. Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets. IEEE Transactions on Software Engineering 44, 2 (Feb 2018), 122--158.

Crossref

Google Scholar

[7]

Sebastiano Panichella, Alessio Gambi, Fiorella Zampetti, and Vincenzo Riccio. 2021. Sbst tool competition 2021. In 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST). IEEE, 20--27.

Crossref

Google Scholar

[8]

Dimitri Stallenberg, Mitchell Olsthoorn, and Annibale Panichella. 2021. Improving Test Case Generation for REST APIs Through Hierarchical Clustering. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 117--128.

Google Scholar

[9]

Maximilian Wohrer and Uwe Zdun. 2018. Smart contracts: security patterns in the ethereum ecosystem and solidity. In 2018 International Workshop on Blockchain Oriented Software Engineering (IWBOSE). IEEE, 2--8.

Crossref

Google Scholar

Cited By

View all

Song SChen JChen TLuo XLi TYang WWang LZhang WLuo FHe ZLu YLi PChristakis MPradel M(2024)Empirical Study of Move Smart Contract Security: Introducing MoveScan for Enhanced AnalysisProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680391(1682-1694)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680391
Semujju SLiu FHuang HXiang YLi XHandl J(2024)Enhancing Fault Detection in Smart Contract Loops Through Adaptive Probabilistic SamplingProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654283(731-734)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654283
Wu SLi ZYan LChen WJiang MWang CLuo XZhou HRoychoudhury APaiva AAbreu RStorey M(2024)Are We There Yet? Unraveling the State-of-the-Art Smart Contract FuzzersProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639152(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639152
Show More Cited By

Index Terms

SynTest-solidity: automated test case generation and fuzzing for smart contracts
1. Software and its engineering
  1. Software creation and management
    1. Search-based software engineering
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Syntest-JavaScript: Automated Unit-Level Test Case Generation for JavaScript
SBFT '24: Proceedings of the 17th ACM/IEEE International Workshop on Search-Based and Fuzz Testing

Over the last decades, various tools (e.g., AUSTIN and EvoSuite) have been developed to automate the process of unit-level test case generation. Most of these tools are designed for statically-typed languages, such as C and Java. However, as is shown in ...
An orchestrated survey of methodologies for automated software test case generation

Test case generation is among the most labour-intensive tasks in software testing. It also has a strong impact on the effectiveness and efficiency of software testing. For these reasons, it has been one of the most active research topics in software ...
More effective test case generation with multiple tribes of AI
ICSE '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings

Software testing is a critical activity in the software development life cycle for quality assurance. Automated Test Case Generation (TCG) can assist developers by speeding up this process. It accomplishes this by evolving an initial set of randomly ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICSE '22: Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings

May 2022

394 pages

ISBN:9781450392235

DOI:10.1145/3510454

General Chair:
Matthew B Dwyer
University of Virginia

This work is licensed under a Creative Commons Attribution International 4.0 License.

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2022

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
668
Total Downloads

Downloads (Last 12 months)292
Downloads (Last 6 weeks)34

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Song SChen JChen TLuo XLi TYang WWang LZhang WLuo FHe ZLu YLi PChristakis MPradel M(2024)Empirical Study of Move Smart Contract Security: Introducing MoveScan for Enhanced AnalysisProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680391(1682-1694)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680391
Semujju SLiu FHuang HXiang YLi XHandl J(2024)Enhancing Fault Detection in Smart Contract Loops Through Adaptive Probabilistic SamplingProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654283(731-734)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654283
Wu SLi ZYan LChen WJiang MWang CLuo XZhou HRoychoudhury APaiva AAbreu RStorey M(2024)Are We There Yet? Unraveling the State-of-the-Art Smart Contract FuzzersProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639152(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639152
Górski T(2024)SmarTS: A Java package for smart contract test suite generation and executionSoftwareX10.1016/j.softx.2024.10169826(101698)Online publication date: May-2024
https://doi.org/10.1016/j.softx.2024.101698
Han QWang LZhang HShi LWang D(2024)Ethchecker: a context-guided fuzzing for smart contractsThe Journal of Supercomputing10.1007/s11227-024-05954-980:10(13949-13975)Online publication date: 13-Mar-2024
https://dl.acm.org/doi/10.1007/s11227-024-05954-9
Elakaş ASözer HŞafak IKalkan K(2024)A systematic mapping on software testing for blockchainsCluster Computing10.1007/s10586-024-04421-727:6(7111-7126)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10586-024-04421-7
Britikov KZlatkin IFedyukovich GAlt LSharygina N(2024)SolTG: A CHC-Based Solidity Test Case GeneratorComputer Aided Verification10.1007/978-3-031-65627-9_23(466-479)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-65627-9_23
Li DWong WPan SKoh LChau N(2023)Automatic Generation of Solidity Test for Blockchain Smart Contract using Many Objective Search and Dimensionality Reduction2023 10th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA59317.2023.00057(418-430)Online publication date: 10-Aug-2023
https://doi.org/10.1109/DSA59317.2023.00057
Olsthoorn Mvan Deursen APanichella A(2022)Guiding Automated Test Case Generation for Transaction-Reverting Statements in Smart Contracts2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME55016.2022.00023(163-174)Online publication date: Oct-2022
https://doi.org/10.1109/ICSME55016.2022.00023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Syntest-JavaScript: Automated Unit-Level Test Case Generation for JavaScript

An orchestrated survey of methodologies for automated software test case generation

More effective test case generation with multiple tribes of AI