A systematic literature review of undiscovered vulnerabilities and tools in smart contract technology

1.1 Scope

Recently, many researchers have published review articles in this field with different types and numbers of vulnerabilities discussed. However, most of them have focused on smart contracts built on Ethereum or based on Ethereum virtual machine (EVM) . In the last few years, other blockchain technologies were proposed by the community to solve some Ethereum technical limitations [5–7]. Most of those blockchains support smart contract building and deploying and offer their languages and concepts.

Unfortunately, smart contracts built on the newer blockchain are still not mature, and there is a huge gap between vulnerability-based security research on them and the number of apps built each day. Therefore, in this article, we try to identify what are the smart contract-building technologies that were deeply studied and those that have less scientific research. We also try to identify the researches that applied machine learning (ML) techniques to find security vulnerabilities in smart contracts.

1.2 Motivation

The motivation for this survey is to help developers, students, and security researchers in the field of smart contract security auditing. The existing surveys do not give a systematic understanding of the different vulnerabilities that can affect multiple types of blockchains. Most reviews either deal with the blockchain attacks that target the network protocol or a specific blockchain technology [8,9], like Ethereum smart contract vulnerabilities.

Therefore, there is a strong requirement for a concise source of systematized information for understanding other blockchain vulnerabilities, their root causes, and the existing tools that can be used to find those vulnerabilities.

1.3 Contributions

We perform a systematic survey of smart contract security vulnerabilities that were identified in every studied blockchain and their detection tools, over the period 2016–2021. Our work helps to provide a thorough understanding of the various blockchain smart contract vulnerabilities. Here are our main contributions:

1. We identify all less-studied smart contract development technologies in terms of security.

2. A deep insight into all discussed smart contract vulnerabilities with their detection and analysis tools.

3. We identify 296 primary studies that discuss smart contract vulnerabilities.

4. We further select 107 studies that meet the inclusion and exclusion criteria we set for the paper screening phase.

5. We conduct an in-depth assessment through a critical analysis of the 107 selected articles and provided a deep insight into the discussed smart contract vulnerabilities.

6. We provide a brief comparison of detection and analysis tools for these vulnerabilities based on five parameters such as type of tool, targeted blockchain smart contract, type of analysis, and availability of the tool to use.

7. We then listed and compared the ML-based solutions according to the blockchain tested on, and the used technique to discover vulnerabilities.

8. We provide a list of the most used vulnerable smart contracts datasets to help future ML-based solutions to train and test their models

1.4 Organization of the article

The organization of this article is as follows: Section 2 provides an overview of the fundamental principles of blockchain technology, including its application and associated terminologies in the current state of the art. Section 3 outlines the methodology employed for the selection and analysis of the researchers. Section 4 presents the findings and outcomes derived from our research, followed by an in-depth discussion of these results in Section 5. This article is concluded in Section 6, offering recommendations for future investigations in related directions.

2.1 Blockchain technology

At its core, a blockchain represents a distributed database or public ledger comprising a comprehensive record of completed transactions or digital events exchanged among participating entities. This decentralized system serves as a foundational framework for ensuring transparency, immutability, and consensus in recording and verifying the integrity of information within the network [10]. The verification and validation of each transaction on the public ledger are achieved through a consensus mechanism involving a majority of the system’s users. Importantly, once data is recorded, it becomes immutable and cannot be altered. This results in a transparent and auditable record of every transaction ever conducted, thereby enhancing the verifiability and traceability of information within the blockchain.

The most well-known example of blockchain technology-related product is Bitcoin and Ethereum. They are also the most contentious since it contributes to the development of an unregulated, multibillion-dollar worldwide market for anonymous trade.

However, the technology behind blockchains is not debatable, and it has operated without a glitch for years and is now being effectively used in both financial and nonfinancial applications.

The four main types of blockchain systems now in use are consortium blockchain, private blockchain, public blockchain, and hybrid blockchain. In a public blockchain, everyone can see all records and participate in the consensus procedure. On the other hand, with a consortium blockchain, the consensus process would only include a select few nodes. Regarding private blockchain, only nodes from a single company would be permitted to participate in the consensus process. A private blockchain is viewed as a centralized network since only one company has complete control over it. Since just a limited number of nodes would be chosen to decide the consensus, the consortium blockchain built by numerous organizations is only partially centralized. A hybrid blockchain is a combination of permissionless and permissioned blockchains. The idea here is that some processes like validating new blocks could be private where only a few known nodes of the same organization or from multiple organizations are able to participate. However, checking transactions and verifying them could be a task allowed for any public node.

2.2 Smart contract and alike

In recent years, the growth of blockchain technology has made it possible to store adaptable programing logic in a decentralized manner. As a result, the idea of smart contracts, which Nick Szabo initially introduced in 1994 [11] and are also known as blockchain contracts, digital contracts, or self-executing contracts, has been revitalized. Smart contracts are self-executing agreements between parties that contain the conditions of the deal. A distributed, decentralized blockchain network contains program codes that are used to create the contracts. Without the requirement for a centralized authority, smart contracts allow transactions to be carried out between unidentified or untrusted parties. Blockchain 1.0, which underpins Bitcoin [12] and other cryptocurrencies, contains the standard properties of decentralization, tamper-resistance, anonymity, and audibility. However, the restrictions of the Bitcoin programing language prevent building contracts with intricate logic.

Bitcoin can only be viewed as the prototype of smart contracts because of its limited capabilities. Newly developed blockchain systems, like Ethereum [13], support the notion of executing user-defined programs on the blockchain and using a Turing complete programing language to create expressive, customizable smart contracts. The EVM executes the stack-based bytecode language used to create the smart contract programs. Solidity is a high-level language that may be used to create Ethereum smart contracts. This language’s source code can then be turned into executable EVM bytecodes. Blockchain 2.0 is known as Ethereum since it is the most widely used platform for creating smart contracts at the moment. However, multiple newly created Blockchains are also offering the same capabilities for building smart contracts. Some of them use even different nominations for smart contracts. Hyperledger blockchain call includes another term “chaincode” to define a set of smart contracts that may have the same characteristics [5]. Solana calls smart contracts programs [7].

2.3 Smart contract vulnerabilities

Smart contracts, like conventional programs created by human developers, are prone to significant security vulnerabilities. Unfortunately, given their involvement in direct financial transactions, any exploited vulnerability within a smart contract can result in substantial financial losses. One notable instance of a widespread smart contract breach is the DAO attack, which led to a staggering loss of 60 million dollars and necessitated a hard fork of the Ethereum blockchain [3].

The heightened risk associated with vulnerable smart contracts, surpassing that of conventional applications, stems from the relative immaturity of the language utilized for their development. This immaturity becomes evident upon analyzing the frequency of annual releases [14]. Furthermore, due to the novelty of smart contract technology, a significant portion of smart contract developers remain unaware of the optimal practices for constructing secure smart contracts. These factors, alongside various other elements, collectively contribute to producing vulnerable smart contracts.

Furthermore, the increasing adoption of smart contracts as a solution for diverse applications [2] has resulted in the escalating complexity of smart contract source code. Consequently, the manual verification of smart contracts has become increasingly impractical due to the extensive number of lines of code involved.

3.1 Research questions

Despite the significant advancements witnessed by the Ethereum blockchain in recent years, this technology still exhibits various technical limitations that hinder its maturity. These limitations encompass concerns such as gas fees and transaction processing capacity. Consequently, numerous research groups have undertaken the development of alternative blockchains to augment this technology and propose more robust blockchain networks. Many of these newly developed blockchains have garnered remarkable community adoption due to the solutions they offer. Moreover, developers are provided with the opportunity to build Dapps on several of these blockchains, facilitating the growth of their respective communities.

Regrettably, there is a lack of comprehensive coverage concerning vulnerabilities in blockchain Dapps, with certain security aspects being inadequately addressed. Hence, this article adopts the method and recommendations outlined by Kitchenham et al. to identify the following research questions. The aim is to enhance developers’ understanding of the vulnerabilities that may arise in their source code, particularly in the context of the Ethereum blockchain and any other blockchain platforms.

RQ1: What are all the studied smart contract vulnerabilities?

RQ2: What are the vulnerabilities that have not been widely studied?

RQ3: What are the developed frameworks and methods to find those vulnerabilities?

RQ4: To what kind of smart contract vulnerabilities ML was applied?

RQ5: What are the used smart contract datasets?

3.2 Information sources and database

Based on a search strategy created to find the pertinent research, the literature search was carried out utilizing several database sources. A thorough automated search was conducted to achieve this goal utilizing five database sources: PubMed, SCOPUS, IEEE Xplore, Science Direct, and ACM. Following the PRISMA criteria at all phases, we created a search strategy to find pertinent literature [17]. Figure 1 shows the article screening and selection procedures that the two authors of this study evaluated.

Figure 1

Number of articles included and excluded through paper screening.

3.3 Search strategy and key terms

The word name smart contract was originally proposed by the Ethereum community to describe the apps that work on the EVM. However, not all the recent Blockchains call those apps a smart contract. Some blockchains like Solana call those apps as Dapps programs. Therefore, we have tried to use both keywords to search for relevant articles.

The following search phrases were used to tailor the search strategy for the five databases:

1. Dapp

2. Smart contract

3. Blockchain application

4. Blockchain programs

5. Vulnerability

Table 1 shows in detail the query that was used for each database search engine.

3.4 Eligibility criteria

Following the initial selection from the aforementioned databases, the next step is the paper screening, which entails evaluating each article’s eligibility in accordance with a variety of inclusion and exclusion criteria, as shown in Table 2, to only retrieve the most pertinent studies that focus on smart contract vulnerabilities detection.

3.5 Selection results

After querying the different articles databases and filtering them for only English articles and those published between 2016 and 2021, we were able to collect 331 research articles. To maintain the quality of the review, all duplicate records were thoroughly checked and removed to reduce the number to 296 articles. In particular, the abstracts of all research articles included in the review process were checked in detail and filtered to ensure their quality and relevance and respect to the different inclusion criteria to keep only 144 of them. Next, we performed a deeper analysis of those articles to filter those that match one of our exclusion criteria. This latest step was able to reduce the number of the included articles to 107 articles. Figure 1 better illustrates this process and the result of each step.

3.6 Data extraction

Selected studies were placed into a data extraction spreadsheet using Google Sheets. The data extracted from the studies were as follows the blockchain technology used in the research, input code type, the smart contract vulnerabilities, methods, and frameworks developed and used to discover them. In addition, the sources of smart contract code are used to test those frameworks and models.

4.1 Bibliometric analysis

In this part, which followed the paper screening process, we evaluated the included articles in terms of their publishers and publication years.

According to Figure 2, more than 51% of the included articles in this research come from IEEE. A total of 36.4% of the articles were published by Scopus and 10.3% by ACM. In addition, both PubMed and Science Direct have equally published 0.9% of the included papers.

Figure 2

Number of primary studies according to their publishers.

It is important to note that security researchers are becoming increasingly concerned about smart contract vulnerabilities. Figure 3 clearly shows the increased number of articles published on this subject each year. As shown in Figure 3, the number of published articles has increased from 6 in 2018 to reach 41 papers in 2021. The decrease in the growth of the number of articles published between 2020 and 2021 could be explained by the global situation caused by the Covid19 and the concentration on finding a solution to the pandemic.

Figure 3

Number of primary studies published over time.

However, the increased adaptation of blockchain technology and the growth of use cases indicate that more researchers will get interested in this subject and more research would be published in the near future [18]. In addition, the increased impact caused by such vulnerabilities is pushing companies to invest more and more in securing their Dapps.

4.2 Technical analysis

In this section, we present all the results of our systematic literature review. The analysis process performed on the selected articles has shown that all the vulnerabilities that have been studied by researchers in the field of blockchain have targeted one of the following blockchains:

• Hyperledger Fabric

  Hyperledger Fabric is an open-source blockchain framework designed for enterprise applications. It offers a modular architecture, enabling customizable consensus protocols and permissioned network configurations. With a focus on scalability, privacy, and interoperability, Hyperledger Fabric provides a robust foundation for building secure and scalable distributed ledger applications in diverse industries [5].

• EOSIO

  EOSIO is a blockchain protocol known for its high throughput and scalability, making it suitable for decentralized applications (Dapps) that require fast and efficient transaction processing. Utilizing a delegated proof-of-stake consensus mechanism, EOSIO achieves consensus through a network of elected block producers, ensuring decentralized governance and robust performance [19].

• VNT Chain

  VNT Chain is a blockchain platform that emphasizes privacy, scalability, and interoperability for decentralized applications (Dapps). It leverages innovative techniques such as zero-knowledge proofs and sharding to ensure data confidentiality and enhance transaction throughput. With its focus on secure and efficient data management, VNT Chain offers a compelling solution for various industry applications [6].

• Ethereum

  Ethereum, a decentralized blockchain platform, revolutionizes the realm of smart contracts and decentralized applications (Dapps). It employs a Turing-complete scripting language and a consensus algorithm called proof of stake to facilitate secure, transparent, and programmable transactions. Ethereum’s vibrant ecosystem and broad developer community contribute to its continuous growth and widespread adoption in various domains [13].

As the name of the same vulnerability could change from one paper to another, we have decided to map the discussed vulnerabilities in those papers to our previous work [20] about vulnerabilities. In addition, for better communication of the results, we have used the same proposed codification.

Table 3 summarizes all the vulnerabilities that have been discussed in our previous work [20]. The first column of this table represents the blockchain platforms where the listed vulnerability could be found according to the selected paper analysis.

Tables 4–7 lists all the vulnerabilities that have been discussed in the selected papers categorized by the type of blockchain that have been studied. Table 4 lists all the discussed vulnerabilities in each selected article that focused on the Hyperledger Fabric blockchain. Table 5 lists all the discussed vulnerabilities in each selected article that focused on the EOSIO blockchain. Table 6 lists all the discussed vulnerabilities in each selected article that focused on the VNT Chain blockchain. Table 7 lists all the discussed vulnerabilities in each selected article that focused on the Ethereum blockchain.

From each article, we have identified what tools were used to discover those vulnerabilities and the datasets that contain those vulnerabilities that have been used to test the used tools. Most articles listed in those tables have deployed their frameworks and models to find those vulnerabilities. However, some of them have only benchmarked the existing tools in the market to check their effectiveness in discovering multiple critical ones.

Unfortunately, not all researchers have published their tools or at least their datasets, which makes their verification a difficult task.

Due to the absence of scientific articles that substantiate or elucidate the employed techniques for all the tools mentioned in the preceding tables, our selection for inclusion in Table 8 is based on two criteria:

1. The tool should be proposed by one of the articles selected for this research.

2. The proposed tool, framework, or technique should be technically implemented and tested and not only theoretically discussed.

The collected tools are presented in Table 8 and classified based on three main characteristics:

• Blockchain technology supporting

• Type of the tools (static, dynamic, or hybrid)

• The type of logic used to find vulnerabilities (artificial intelligence or algorithm)

• Source code availability

5.1 What are all the studied smart contract vulnerabilities?

According to our analysis performed on the selected articles, all smart contracts were developed utilizing the Hyperledger Fabric, EOSIO, Ethereum, or VNT Chain as the underlying platforms. Consequently, the classification of smart contract vulnerabilities often aligns with the specific Blockchain platforms on which they manifest. Nevertheless, certain vulnerabilities exhibit technical similarities and possess a more generic nature, enabling their occurrence across multiple types of blockchains.

Figure 3 illustrates the frequency of articles that have presented a comprehensive analysis of a specific vulnerability. As you can see, only 3 of 72 vulnerabilities have never been discussed in one of the selected articles.

In addition, multiple articles have extensively investigated vulnerabilities such as SWE-107, SWE-116, and SWE-101.

According to our research results, the SWE-107 vulnerability has emerged as the most prevalent vulnerability within the selected article. Its prominence can be attributed to its association with the infamous DAO hack, which served as a catalyst for the Ethereum hard fork. The widespread interest in addressing this vulnerability has spurred a substantial body of research, characterized by a multitude of scholarly articles dedicated to its investigation and mitigation. In addition, our current understanding suggests that the prevalence of this vulnerability in smart contracts can be attributed, in part, to the similarity of use cases observed across various types of smart contracts, such as tokens and nonfungible tokens (NFTs).

Moreover, some of the articles that addressed this vulnerability have even divided it into four subtypes: the fallback function-based re-entrancy, the create-based reentrancy, the cross-function reentrancy, and the delegated call Reentrancy. However, this categorization is not very popular among the papers that have discussed this vulnerability.

At some point in the development of any type of application, knowing the time or the date at which an operation is executed is mandatory. Therefore, a common behavior from developers was to use the block information as a source of such information. However, this behavior has created a popular vulnerability codified as SWE-116. Unfortunately, the wide use of this technique to know the time has made this vulnerability very common in smart contracts, which attracted more security researchers to analyze it and build frameworks to find it.

The SWE-101 vulnerability concern the integer over/underflow. This vulnerability is very common among programing languages especially those that are very close to the machine language. Therefore, abording this subject might be easily compared to more complex and blockchain-specific vulnerabilities.

5.2 What are the vulnerabilities that have not been widely studied?

According to the results of our research, the average number of articles for each vulnerability is about 9.4 (Figure 4). This result means that every 10 vulnerabilities have at least been cited in 94 articles. On the basis of this result, we have decided that a vulnerability is considered poorly studied if it has nine papers or fewer that cite it.

Figure 4

Number of papers that have discussed each vulnerability.

Therefore, according to these criteria, we have found that more than 71% of the existing vulnerabilities were not widely discussed by research in this field. Those vulnerabilities are as follows: SWE-109, SWE-139, SWE-141, SWE-142, SWE-132, SWE-145, SWE-151, SWE-118, SWE-159, SWE-119, SWE-121, SWE-144, SWE-111, SWE-124, SWE-125, SWE-150, SWE-103, SWE-110, SWE-129, SWE-147, SWE-162, SWE-163, SWE-165, SWE-166, SWE-167, SWE-168, SWE-169, SWE-170, SWE-171, SWE-175, SWE-176, SWE-102, SWE-123, SWE-127, SWE-148, SWE-149, SWE-152, SWE-154, SWE-155, SWE-164, SWE-172, SWE-173, SWE-122, SWE-134, SWE-146, SWE-153, SWE-156, SWE-157, SWE-158, SWE-160, SWE-161, SWE-174, SWE-117, SWE-133, and SWE-135.

This result could be explained by the fact that some vulnerabilities are either not very dangerous compared to others or they are specific to blockchains that are themselves not widely used or simply not known enough to become subject to research.

The SWE-103 vulnerability is related to a floating pragma that may lead unintentionally to include compiler-based vulnerabilities if compiled by older or newer versions of solidity. This issue is a good example of a specific vulnerability that could only be found on Ethereum-based smart contracts, especially those built by solidity. Moreover, most security research works do not give this vulnerability a big intention as its impact could in some situations be considered negligible.

Three of the vulnerabilities listed in this research were never studied in all the selected articles, which are SWE-117, SWE-133, and SWE-135. Both SWE-117 and SWE-133 are vulnerabilities related to cryptography implementation issues. Both vulnerabilities are not very popular as that implementation could be considered a rare use case of smart contacts.

The SWE-135 vulnerability is related to code left in the smart contract with no effects. This vulnerability is very dangerous that could lead to financial losses. Unfortunately, no article has ever discussed these types of vulnerabilities among the selected articles.

In addition, only less than 24% were massively studied by the selected articles. Those vulnerabilities are SWE-107, SWE-116, SWE-101, SWE-137, SWE-114, SWE-140, SWE-120, SWE-104, SWE-128, SWE-113, SWE-112, SWE-115, SWE-126, SWE-138, SWE-100, SWE-136, and SWE-143.

5.3 What are the developed frameworks and methods to find those vulnerabilities?

As the adoption of smart contract technology proliferates among companies and individuals, the daily deployment rate of smart contracts continues to escalate. Furthermore, a growing number of individuals have harnessed smart contracts to address various challenges [135]. This widespread adoption across multiple domains has concurrently augmented the complexity of smart contracts, rendering manual vulnerability discovery a daunting and time-intensive task.

For this reason, many security researchers have proposed tools and frameworks to find those vulnerabilities using different concepts and techniques.

To address this inquiry, we formulated a selection criterion for our analysis. We specifically targeted articles that introduced novel frameworks, providing comprehensive insights into their internal workings. Consequently, articles primarily focused on framework comparisons were excluded from our examination. Moreover, articles solely presenting theoretical concepts lacking technical implementations were also disregarded.

Table 8 presents a comprehensive compilation of frameworks detailed in the selected articles, encompassing the employed vulnerability detection techniques as well as the availability of corresponding source code or binary. Our findings reveal a notable overlap between the vulnerability detection techniques utilized in traditional applications and those employed in smart contracts. This means that techniques like symbolic execution [120], pattern matching [34], data flow analysis [62], control flow [50] analysis, fuzzing, and many others [136,137] are used to find smart contract vulnerabilities.

These frameworks can be categorized into three primary types: static analysis-based frameworks, dynamic analysis-based frameworks, and hybrid analysis-based frameworks [136,137]. Furthermore, certain frameworks were specifically developed to address distinct vulnerabilities, such as re-entrancy (SWE-107), while others aimed to detect a range of vulnerabilities. Regrettably, only 31 of 69 frameworks have made their source code publicly available, facilitating verification and utilization in future research endeavors.

5.4 To what kind of smart contract vulnerabilities the ML was applied?

Likewise, in classic applications, ML technology was also used in vulnerability research in smart contracts. Ziyuan et al. [104] have used ML technology to discover vulnerabilities like the re-entrancy (SWE-107) and used block values as a proxy for time (SWE-116). Daojing et al. [106] have also used ML technology to find multiple vulnerabilities like SWE-107, SWE-116, SWE-148, SWE-140, SWE-100, and SWE-101. However, the most relevant research was the one performed by Momeni et al. [124] where they used four different machine-learning techniques to build multiple models that are capable to detect specific vulnerabilities.

The resulting model developed by this team was capable to find 16 different vulnerabilities with an average accuracy of 95%. What makes this research very relevant is the number of vulnerabilities that could be discovered with this level of accuracy.

Table 9 presents the result of our research:

However, according to our results, we have discovered that most ML implementations have targeted the Ethereum virtual machine-based smart contracts. Moreover, no machine-learning model has been developed to target Hyperledger fabric or any other private or public Blockchains.

5.5 What are the used Smart contract datasets?

Most of the used smart contracts came from the Etherscan platform. This platform offers a free service of continuous Ethereum blockchain scrapping to allow reading information. In addition, the platform allows developers to validate their smart contracts by submitting their source code and verifying it to add credibility to their smart contracts. This verification process helps increase the smart contract source code dataset. However, the dataset offered by Etherscan is not labeled in terms of vulnerable functions. About 35% of the researchers in the selected articles have used this dataset.

However, for labeled smart contracts source code, some researchers have used Smart bugs or Solidifi-benchmark or both to check their models and algorithms. Solidifi-benchmark has used 9369 defects from 7 distinct problem categories, including reentrancy, timestamp dependence, unchecked transmit, TOD, integer overflow/underflow, and usage of tx.origin, to build a dataset of flawed contracts in the SolidiFI-benchmark repository using SolidiFI.

The Smartbugs dataset also offers hundreds of labeled vulnerable smart contracts that can be used to train or test models.

Unfortunately, a big part of the selected articles did not specify the used dataset for their research and others have performed manual collection of vulnerable smart contracts using GitHub or any website citing vulnerable smart contracts or even CTFs like Ethernaut.