Smart Contract Security Put to the Test: Ensembles Beat Deep Learning, but Active Learning Falls Short

Smart contracts have quietly become the backbone of the decentralized economy, executing billions of dollars in transactions without human intermediaries. But the same self-executing code that makes blockchain applications so powerful also makes them dangerously fragile. A single overlooked bug can drain funds, corrupt contract state, or destabilize an entire network. A new study published in Frontiers in Blockchain tackles a deceptively simple question: when it comes to training machines to spot vulnerable smart contracts, does the cleverness of the data-selection strategy matter more than the raw power of the classifier itself?

The research, led by Gabriel Avelino Sampedro of Lyceum of the Philippines University and Universitas Brawijaya together with colleagues from institutions in China, Indonesia, and the Philippines, examined whether uncertainty-based active learning can slash the expensive labeling burden that plagues smart contract security research. Labeling a smart contract as vulnerable or secure is not a task that can be crowdsourced cheaply; it typically demands expert analysis of bytecode, abstract syntax trees, and source-level patterns. If an algorithm could learn which unlabeled contracts are most worth an expert’s time, the economics of building detection systems might change dramatically.

To find out, the team turned to the BCCC-VulSCs-2023 dataset, a collection of 36,670 smart contracts described by 73 raw feature columns, including bytecode-derived opcode frequencies, abstract syntax tree structural measures, and keyword counts, each tagged with a binary vulnerability label. The raw data required careful preparation. Constant and irrelevant columns were stripped away, the class imbalance was corrected using SMOTE oversampling, the data were split 80/20 into training and test partitions, and SelectKBest with the ANOVA F-statistic was applied to retain the 14 most informative features, which were then normalized with StandardScaler.

The feature selection itself yielded a striking insight into what makes a contract dangerous. The Solidity keywords “view,” “emit,” and “external” produced the largest F-statistics, followed by bytecode opcode-frequency features such as weight_sums_MSTORE and weight_sums_DIV, and AST-derived structural measures like node counts and exported symbol counts. All 14 features remained statistically significant after Benjamini-Hochberg false discovery rate correction, and 13 of them were reselected in every one of 25 cross-validation folds, a level of stability that suggests the signal is real rather than a statistical fluke. The effect sizes were small, however, ranging from 0.015 to 0.057, indicating that no single feature dominates and that vulnerability detection depends on a multivariate pattern rather than any one smoking gun.

The experimental design was deliberately rigorous. Three ensemble classifiers, Random Forest, XGBoost, and CatBoost, were each evaluated under three conditions: an active learning regime that began with 1,000 randomly sampled labeled instances and then queried the 4,000 most uncertain samples per iteration across 10 rounds; a random-sampling control that followed the identical labeling schedule but picked samples blindly; and a fully supervised baseline trained on the complete training set. Against these, the team pitted three deep learning architectures, LSTM, DNN, and GRU, trained on the same preprocessed, feature-selected data and evaluated with accuracy, precision, recall, and F1-score on a held-out test partition.

The headline numbers tell a story of two winners and a humbling result. Random Forest with active learning reached 73.7 percent test accuracy and CatBoost with active learning hit 73.9 percent, comfortably outperforming XGBoost with active learning at 69.1 percent and all three deep learning baselines, which landed between 66.7 and 68.2 percent. The confusion matrices confirmed that Random Forest and CatBoost made fewer misclassifications than their competitors under both active learning and random sampling. The deep learning models converged stably during training with no severe overfitting, but their absolute performance suggested that the flattened, non-sequential input representation may have limited their ability to exploit the structured feature set.

But here is where the study delivers its most provocative finding: the active learning advantage simply did not materialize. Across all three classifiers, test accuracy differed by less than 0.3 percentage points between the uncertainty-based and random-sampling conditions, and in two of the three cases, XGBoost and CatBoost, random sampling actually edged out active learning at the final iteration. Both approaches approached the accuracy of the fully supervised baseline, which is perhaps unsurprising given the labeling budget: 41,000 of 43,062 available training labels, or 95.2 percent of the entire training set. When you have labeled nearly everything, the choice of what to label next barely matters.

The authors are careful to frame this as a boundary condition rather than a refutation. Active learning is theorized to shine brightest at small labeling budgets, where each labeling decision carries maximum weight and the model’s uncertainty signal can steer the learner away from redundant examples. At 95 percent coverage, the query pool has been so thoroughly drained that uncertainty sampling and random sampling converge on essentially the same training distribution. The study’s design, which fixed the budget at a single point rather than sweeping across a range of budgets, cannot rule out a genuine advantage at, say, 5 or 10 percent coverage, where the labeling savings would matter most in practice.

The implications for the blockchain security community are nonetheless substantive. First, the dominance of Random Forest and CatBoost over deep learning baselines suggests that for structured, tabular representations of smart contract features, tree-based ensembles remain the tool of choice, and practitioners should not assume that neural architectures automatically outperform classical methods. Second, the finding that random labeling performs as well as uncertainty sampling at high budgets offers a practical shortcut: teams building vulnerability detectors with abundant unlabeled contracts but limited expert time may not need sophisticated query strategies if they can afford to label most of their data anyway. Third, the feature stability analysis provides a reproducible foundation for future work, identifying which code-level signals consistently separate vulnerable contracts from secure ones.

The road ahead, as the authors outline, involves testing active learning across a spectrum of smaller labeling budgets, extending the evaluation to additional smart contract datasets, and exploring richer code representations, including transformer-based models that can read source code as language rather than as flattened feature vectors. They also flag the need to examine nonlinear feature interactions and model-specific importance measures that the univariate ANOVA approach cannot capture. Until then, the study stands as a valuable corrective to hype: in the race to automate smart contract security, the humble ensemble classifier wins, and the fancy sampling strategy, at least at this budget, adds nothing that a coin flip could not achieve.

Subject of Research: Machine learning methods for detecting vulnerabilities in blockchain smart contracts

Article Title: Comparative evaluation of active learning, random sampling, and deep learning for smart contract vulnerability detection

Article References: Sampedro, G. A., Yanhui, C., Caballero, A., Putra, M. A. P., Aditya Karna, N. B., & Indriati (2026). Comparative evaluation of active learning, random sampling, and deep learning for smart contract vulnerability detection. Frontiers in Blockchain, 9, Article 1882027. https://doi.org/10.3389/fbloc.2026.1882027

Image Credits: AI Generated

DOI: 10.3389/fbloc.2026.1882027

Keywords: smart contracts, blockchain security, active learning, vulnerability detection, machine learning, Random Forest, CatBoost, XGBoost, deep learning, LSTM, SMOTE, feature selection