Smart Contract Auditing Tools Reviewed: Pros, Cons, And The Need For Manual Checks

Web3 businesses lost $1.9 billion in 2023. Nearly $400 million of that amount came from smart contract vulnerabilities, such as flashloan and reentrancy attacks. Of all exploited projects that year, 90% had never undergone any form of audit. The data proves the effectiveness of having your code reviewed and analyzed for security by a knowledgeable third-party auditor, as it significantly lowers the chances of exploits in audited code.

But as we dive deeper into the data, there’s a problem – not all audits are equally effective. Of all smart contracts with relevant audits, 26% were breached in reentrancy attacks, and 11% suffered flash loan attacks.

Why do some audits, despite targeting the exploited scope, fail to protect the contracts? Apart from the obvious reasons–team expertise, depth of manual review, and operational excellence–the particular choice of smart contract auditing tools makes a huge difference in the audit outcome.

As a blockchain auditor with the lowest post-audit exploit rate on the market, we at Hacken want to offer our expert review of smart contract auditing tools to help Web3 developers deploy more secure code.

Smart Contract Audit Tools

Many developers and project founders use automated tools like Slither, Solgraph, Mythril, Echidna, and MythX. While powerful and efficient, these are often seen as a one-size-fits-all solution to the complex and nuanced challenges of smart contract security.

A more effective approach is to break down the audit process into functional pieces and apply the right tool for a specific job – static and dynamic analysis, fuzzing, mutation testing, etc.

Static & Dynamic Analysis Tools

Static analysis tools review code without running it, spotting potential security and coding flaws by examining code structure and syntax. Dynamic analysis tests software in runtime, detecting vulnerabilities, memory leaks, and performance problems during specific operational conditions.

Prominent static analysis tools:

• Slither
• Mythril
• MythX
• Securify2
• Manticore
• Fuzzinglab’s Octopus

1. Slither

Slither is an open-source static analysis tool designed for Solidity & Vyper smart contracts. It scrutinizes Solidity and Vyper code to detect known vulnerabilities, such as reentrancy, boolean equality, and unused return values. 

Advantages

• Comes with 92 built-in detectors and even offers custom detectors. This flexibility allows users to tailor their analysis specifically for Solidity and Vyper smart contracts, focusing on particular vulnerabilities of interest.
• Provides Inheritance Graph generation for any given contract. This feature is especially beneficial for auditors, as it aids in comprehensively understanding the system’s structure and interactions.
• Call Graph, which maps out the interactions and calls between different functions within a contract, offering deeper insight into its operational flow.
• Fast Execution.

Disadvantages

• Limited solely to Solidity and Vyper smart contracts.
• There will be a bunch of false positives identified as an issue.

2. Mythril

Mythril is a free and open-source smart contract security analyzer for EVM bytecode. It uses symbolic execution to detect a variety of security vulnerabilities. 

Advantages

• Supports Ethereum, Hedera, Quorum, Vechain, Roostock, Tron and other EVM-compatible blockchains.

Disadvantages

• It’s not customizable, and users can’t create their own detectors.

3. MythX

Mythx is a paid version of Mythril, a smart contract security analyzer for EVM bytecode.

Advantages

• MythX performs a thorough analysis, combining static analysis, dynamic analysis, and symbolic execution.
• Provides a more user-friendly interface compared to some other tools. The generated reports can be accessed from their official website.

Disadvantages

• It’s a paid service.
• Users can not write their own detectors; it’s limited.

4. Securify2

Securify v2.0 is a sophisticated static analysis tool specifically designed for auditing Ethereum smart contracts written in Solidity. It employs advanced context-sensitive analysis techniques to detect a wide range of security vulnerabilities.

Advantages

• It comes equipped with 32 built-in detectors.
• Implements novel context-sensitive static analysis.

Disadvantages

• Analyzes contracts written in Solidity 0.5.8 or above.
• Primarily focuses on static analysis, which might not cover all types of security vulnerabilities that could be revealed through dynamic analysis or symbolic execution.

5. Manticore

Manticore is a symbolic execution tool designed for analyzing and testing Ethereum smart contracts. It abstractly explores the contract’s state space, uncovering bugs that cannot be detected through traditional testing methods, and it is for Ethereum-based applications. 

Advantages

• The adaptable state management lifecycle enables its utilization in analyzing various software types, ranging from Ethereum smart contracts to Linux ELF binaries and WASM modules.
• Free to use.

Disadvantages

• It requires significant memory resources and may exhibit slow performance.

6. Fuzzinglab’s Octopus

Octopus serves as a security analysis framework designed for both WebAssembly modules and Blockchain Smart Contracts. The primary goal of Octopus is to facilitate the examination of closed-source WebAssembly modules and smart contracts bytecode, enabling a deeper understanding of their internal behaviors.

Advantages

• It provides Control Flow Graphs (CFG) and Call Flow Graphs that offer visual representations of program execution and function calls.
• Free to use.

Disadvantages

• Require significant computing power, potentially causing delays for users with limited hardware resources.

Fuzzing Tools

Fuzzing involves the automated input of unexpected or random data into a software system to identify vulnerabilities or weaknesses. This technique helps uncover unforeseen issues that might not be apparent through traditional testing methods.

Common fuzzers:

• Echidna
• Hacken-Fuzz
• Foundry Fuzz
• ChainFuzz
• Harvey
• sFuzz

1. Echidna

Echidna stands out as a sophisticated Ethereum smart contract fuzzer, leveraging Haskell to conduct property-based testing. It’s designed to find vulnerabilities by falsifying user-defined predicates or Solidity assertions through grammar-based fuzzing campaigns tailored to a contract’s Application Binary Interface (ABI). 

Advantages:

• Tailored Input Generation: Echidna generates inputs specifically designed for your code, enhancing the efficiency of bug detection.
• Advanced Bug Detection Techniques: With optional corpus collection and mutation, alongside coverage guidance, Echidna digs deeper to uncover elusive bugs.
• Broad Integration: Before starting a fuzzing campaign, Echidna utilizes Slither to gather valuable contract information, streamlining the testing process.

Disadvantages:

• Vyper support is limited
• Limited library support for testing

2. Hacken-Fuzz

Hacken-Fuzz is a proprietary automated tool offered exclusively as part of Hacken’s cybersecurity services. It streamlines the fuzz target setup process for a variety of programming languages such as C, C+, Rust, Go, JavaScript, JVM, Swift, and Python. Utilizing Hacken’s robust infrastructure, which includes the capability to operate 50 servers concurrently based on OSS-Fuzz, Hacken-Fuzz significantly enhances the efficiency and scale of fuzz testing efforts. Its primary aim is to mitigate the manual, resource-heavy tasks involved in preparing for fuzz testing—tokenizing targets, adhering to specific formatting, and cloud uploads.

Advantages:

• Automation of Fuzz Target Setup: Minimizes manual labor and accelerates the fuzzing preparation process.
• Broad Programming Language Support: Adaptable for a wide array of software development projects.
• Scalable Testing Capability: Leverages extensive server infrastructure for comprehensive testing.
• Effective Vulnerability Detection: Increases detection rate of critical security flaws important for blockchain integrity and security.

Disadvantages:

• Resource Demands: High computing resources are necessary for optimal operation, requiring substantial infrastructure.
• Time Investment: The process can be lengthy, particularly for analyzing complex or extensive protocols.
• Complex Bug Detection Limitations: Some multi-step or intricate vulnerabilities may elude detection.
• Error Type Detection Limits: Might not catch all logic or state-related errors, especially in complex multi-threaded environments.

Application within Hacken Services

Offered exclusively through Hacken’s cybersecurity services, Hacken-Fuzz is tailored to enhance the security of blockchain protocols and smart contracts. It’s pivotal in simulating user behaviors, identifying hidden vulnerabilities, and tackling the inherent complexities of smart contracts and blockchain protocols. Although powerful, Hacken-Fuzz complements a holistic security audit approach, necessitating expert review and verification to address all potential security concerns comprehensively. 

3. Foundry’s Fuzzing

Foundry Fuzzing is a software testing technique that involves automatically generating and injecting malformed or unexpected inputs into a program to identify vulnerabilities, errors, or unexpected behaviors. It systematically explores input spaces, aiming to discover security flaws in applications, libraries, or protocols.

Advantages

• Effective in discovering unknown vulnerabilities in software by generating random or unexpected inputs. Traditional testing methods may not cover all possible inputs and edge cases, but fuzzing helps test a wide range of inputs, making it more likely to identify hidden issues.
• Offers multiple testing methods, such as unit testing, fuzz testing, invariant testing, and differential testing.
• Free to use.

Disadvantages

• It is not as widely documented as popular approaches like Truffle/Hardhat, making it potentially challenging for developers to adopt and navigate.
• Creating Solidity contracts explicitly for testing in Foundry can pose challenges and may require a significant time investment.

4. Harvey

Harvey uses a lightweight test-generation approach that effectively detects bugs and security vulnerabilities in smart contracts.

Advantages:

• Greybox fuzzing effectively detects bugs and security vulnerabilities in smart contracts.
• Harvey extends greybox fuzzing with techniques to predict new inputs likely to cover new paths or reveal vulnerabilities.
• Harvey fuzzes transaction sequences in a targeted and demand-driven manner, significantly increasing effectiveness in achieving high coverage and detecting vulnerabilities.

Disadvantages:

• Results may not generalize to all smart contracts or program types.
• Potential for systematic errors in setup risks internal validity.
• Ensuring evaluation measures what it claims challenges construct validity.

Other available fuzzers are ChainFuzz and sFuzz. Used via Docker, ChainFuzz checks custom property violations, violated assertions, and arithmetic under and overflows and returns the results in a JSON file. It’s best for quick Ethereum transactions fuzzing. sFuzz is an adaptive fuzzer designed for smart contracts on the Ethereum platform, utilizing a combination of AFL fuzzer strategy and a lightweight multi-objective adaptive strategy to efficiently test and discover vulnerabilities in Solidity code, demonstrating superior speed and effectiveness compared to existing tools.

Mutation Testing Tools

Mutation testing introduces small, controlled changes (mutations) to the source code to evaluate the effectiveness of the test suite. It helps identify areas where tests may be inadequate or missing, ultimately enhancing the thoroughness of test coverage.

Common mutation testers:

• SuMo
• Certora Gambit
• Vertigo

1. SuMo

SuMo is a mutation testing tool specifically designed for smart contracts written in the Solidity language. It serves to evaluate the robustness of test suites by injecting small faults, or “mutations,” into the source code. SuMo checks if these changes are detected by the test suites, essentially providing an assessment of the tests. This process identifies weak points in the tests and enhances overall test quality.

Advantages: 

• SuMo provides a wide range of mutation operators, both traditional and Solidity-specific, which can be enabled or disabled based on the needs of the project.
• It supports all Truffle, Hardhat, Brownie, and Foundry projects

Disadvantages:

• Running mutation tests consumes a considerable amount of time, especially when dealing with moderately sized projects

Other mutation testers include Vertigo by Joran Honig and Gambit by Certora. Vertigo offers a valuable tool for enhancing the quality and effectiveness of test suites in smart contract development by generating mutants and checking if the tests can detect potential bugs. Moreover, it gives an objective metric called mutation score to find out the effectiveness of test suites at finding errors. Gambi, an open-source mutation generator for Solidity, offers similar functionality. It’s general-purpose and can be used with various verifiers or testing tools. 

Code Coverage Tools

A code coverage tool is a crucial software quality assurance tool that measures the extent to which the source code of a program is executed when a particular test suite runs. This tool helps developers identify parts of their code that have not been tested, which could contain potential errors or security vulnerabilities. By ensuring all parts of the code are tested, developers can improve the reliability and security of their software.

Notable code coverage instruments:

• Wasmcov
• Solidity-Coverage (Solcover)
• Istanbul.js
• LLVM-cov

1. Wasmcov

Wasmcov is a specialized code coverage tool designed specifically for WebAssembly (Wasm) environments. It’s a Rust library accompanied by a binary that automates coverage analysis of Wasm executables. Leveraging advanced LLVM functionality, Wasmcov aims to overcome the limitations present in current Wasm compilation processes. Its primary goal is to provide developers with precise and efficient code coverage data, crucial for detecting untested code segments and potential vulnerabilities in smart contracts and other Wasm based applications. Wasmcov was developed by Hacken during the audit of L1 protocols Radix and NEAR.

Key features of Wasmcov:

• Targeted for Wasm: Tailored specifically for the Wasm environment, addressing the gap in code coverage tools for this platform.
• Accurate Coverage Data: Provides precise code coverage measurement directly on the target system, avoiding discrepancies between host and target environments.
• Automation of Test Coverage: Facilitates automated test coverage analysis, enabling developers to identify and address potential vulnerabilities efficiently.
• Support for Secure Coding: Helps identify untested code segments that could harbor security vulnerabilities, thus enhancing the security of Wasm based applications.
• Open Source: Being open-source, it can be easily integrated into any Wasm application, encouraging widespread adoption and community contributions.

2. Solidity-Coverage 

Solidity-Coverage, or Solcover, is a tool for measuring the test coverage of Ethereum smart contracts written in Solidity, helping developers identify untested paths in their code.

Advantages:

• Automated Instrumentation & Detailed Reporting: Simplifies test coverage tracking by modifying source code to include execution markers and generates comprehensive reports to identify untested parts of the contract.
• Advanced Configuration Options: Offers a variety of advanced options for tailored testing experiences, including target testing, custom paths, and JSON object generation that maps which mocha tests hit which lines of code.

Disadvantages:

• Limited to Solidity: Its functionality is specifically designed for Solidity smart contracts, limiting its use to Ethereum blockchain development.
• Depreciation of Advanced Options: Many advanced configurations are now depreciated.

More general code coverage tools include Istanbul.js and LLVM-cov. Istanbul is a JavaScript test coverage tool that computes statement, line, function, and branch coverage for Node.js programs. Part of the LLVM project, LLVM-cov offers coverage analysis for programs compiled with LLVM, enabling detailed insight into test coverage and code quality.

The Importance Of Manual Review In Smart Contract Auditing

Slither, Solgraph, Mythril, Echidna, and MythX have revolutionized auditing by swiftly identifying common vulnerabilities. While these automatic tools provide valuable insights and save time in manual analysis, they cannot be a substitute for manual review.

According to recent IEEE research, current security tools can only detect 8-20% of exploitable bugs. This highlights a notable deficiency in automated bug-finding capabilities. Particularly challenging are issues such as asset lock, logical errors, or oracle manipulations, which prove to be extremely difficult for security analyzers alone to detect.

Human involvement is very important in dealing with problems that go beyond what automatic tools can handle. People bring a nuanced understanding to challenges such as uncommon vulnerabilities, logical errors, and human and economic factors. Although automatic tools are good at finding common coding mistakes and standard vulnerabilities, they don’t have the contextual insights and broader considerations needed for a complete approach to smart contract security

Final Thoughts

Automatic tools like Slither and Mythril significantly aid in smart contract auditing by identifying vulnerabilities and simulating attacks. Yet, their effectiveness is limited; they can’t catch uncommon vulnerabilities, logical errors, human factors, or economic manipulations. These areas demand human expertise and a nuanced understanding of the ecosystem. While automatic tools excel at spotting common coding mistakes and known vulnerabilities, a comprehensive security strategy must blend these tools with manual review and contextual analysis.

FAQs

Follow @hackenclub on 𝕏 (Twitter)