Abstract
Background: Executing, verifying and enforcing credible transactions on permissionless blockchains is done using smart contracts. A key challenge with smart contracts is ensuring their correctness and security. Several test input generation techniques for detecting vulnerabilities in smart contracts have been proposed in the last few years. However, a comparison of proposed techniques to gauge their effectiveness is missing. Aim: This paper conducts an empirical evaluation of testing techniques for smart contracts. The testing techniques we evaluated are: (1) Blackbox fuzzing, (2) Adaptive fuzzing, (3) Coverage-guided fuzzing with an SMT solver and (4) Genetic algorithm. We do not consider static analysis tools, as several recent studies have assessed and compared effectiveness of these tools. Method: We evaluate effectiveness of the test generation techniques using (1) Coverage achieved - we use four code coverage metrics targeting smart contracts, (2) Fault finding ability - using artificially seeded and real security vulnerabilities of different types. We used two datasets in our evaluation - one with 1665 real smart contracts from Etherscan, and another with 90 real contracts with known vulnerabilities to assess fault finding ability. Result: We find Adaptive fuzzing performs best in terms of coverage and fault finding over contracts in both datasets. Conclusion: However, we believe considering dependencies between functions and handling Solidity specific features will help improve the performance of all techniques considerably.
Original language | English |
---|---|
Title of host publication | Proceedings of the 15th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) |
Place of Publication | New York, NY, USA |
Publisher | Association for Computing Machinery, Inc |
Number of pages | 11 |
ISBN (Electronic) | 9781450386654 |
DOIs | |
Publication status | Published - 11 Oct 2021 |
Event | 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2021 - Online Duration: 11 Oct 2021 → 15 Oct 2021 Conference number: 15 https://conf.researchr.org/home/esem-2021 |
Publication series
Name | ESEM '21 |
---|---|
Publisher | Association for Computing Machinery |
Symposium
Symposium | 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2021 |
---|---|
Abbreviated title | ESEM 2021 |
Period | 11/10/21 → 15/10/21 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- Input Generation
- Fault Seeding
- Blockchain
- Fuzzer
- Smart Contract
- Constraint Solver
- Genetic Algorithm
- Ethereum