TabNet brings interpretable attention to miRNA classification, but classical models still lead

Deep learning models are increasingly applied to genomic data, but a new study in BMC Bioinformatics suggests that when it comes to classifying disease from microRNA expression profiles, the newest attention-based architecture does not beat the competition on accuracy. Instead, its main advantage lies elsewhere: it produces sparse, instance-level explanations of which miRNAs drove each prediction as an inherent part of its forward computation.

The study, led by Hilal Haznedar with Gözde Özsert Yiğit and Bülent Haznedar, evaluated TabNet, an attentive interpretable tabular learning architecture, on two publicly available miRNA expression datasets: a cervical cancer cohort of 58 samples with 714 miRNAs and an Alzheimer’s disease cohort of 70 samples with 503 miRNAs. The work addresses a fundamental challenge in computational biomedicine, since raw RNA sequencing read counts are high-dimensional, over-dispersed and heteroscedastic, and require careful preprocessing before machine learning can be applied.

Following the MLSeq framework, the researchers applied DESeq median-ratio normalisation, near-zero variance filtering, supervised gene ranking based on differential expression analysis, and the DESeq variance-stabilising transformation. Gene subsets corresponding to 5, 10 and 30 percent of the post-filtering feature space were evaluated, so that performance could be examined across different levels of feature selection.

TabNet was benchmarked against eight baselines spanning four learning paradigms: a 1D convolutional neural network, a long short-term memory network, Extreme Gradient Boosting, Random Forest, ElasticNet-regularised logistic regression, a multi-layer perceptron, the prior-fitted transformer TabPFN, and the inherently interpretable Explainable Boosting Machine. Each configuration was assessed over 50 independent random train/test splits at a 70/30 ratio.

A key strength of the evaluation is its strict leakage-free protocol. All preprocessing, hyperparameter selection and early stopping relied exclusively on inner validation subsets of the training partitions, so no information from the test data could influence model fitting. All metrics are reported as mean plus or minus one standard deviation, giving a realistic picture of the variability expected in such small cohorts.

Under this protocol, a group of four methods reached a statistically indistinguishable accuracy ceiling. On the cervical cancer dataset, ElasticNet led at 95.06 percent (±4.66), followed by EBM at 94.94 percent (±4.60), TabPFN at 94.71 percent (±4.64) and Random Forest at 94.35 percent (±5.82). On the Alzheimer’s disease dataset, TabPFN led at 70.86 percent (±10.26), with EBM at 70.67 percent (±12.67) and ElasticNet at 70.48 percent (±9.76) close behind.

TabNet itself reached 85.18 percent (±11.10) on the cervical cancer data and 64.38 percent (±11.23) on the Alzheimer’s data at its best configurations, placing it below the top tier. The authors argue, however, that this moderate accuracy cost buys something the other methods do not provide: sequential attention that yields sparse, instance-level miRNA attributions as part of the prediction itself, rather than as a post-hoc explanation layered on top.

Supplementary analyses added important caveats. Self-supervised pre-training conferred no significant benefit at these cohort sizes, and results were robust to whether features were ranked by supervised differential expression or by unsupervised variance. Notably, instance-level attention patterns varied substantially across the random splits, which the authors say supports their use as illustrative rather than confirmatory evidence about which miRNAs matter.

The broader conclusion is a cautionary one for the field. In rigorously leakage-controlled small-cohort miRNA classification, strongly regularised classical models, glass-box additive models and prior-fitted transformers converge on the same accuracy ceiling, and no evaluated method, including attention-based deep learning, surpasses it. The authors argue for protocol-first evaluation and calibrated claims when deep tabular models are applied to small transcriptomic cohorts, where differences between methods can easily be overstated.

For researchers working with similar datasets, the study offers a practical template: a leakage-free pipeline, multiple baselines across paradigms, and honest reporting of variability. It also suggests that the choice of model should depend on the goal, since an interpretable deep architecture may be worth its accuracy cost when embedded, per-sample explanations are a priority, while classical methods remain the safest route to maximum predictive accuracy on small miRNA cohorts.

Subject of Research: Benchmarking the TabNet deep learning architecture against classical machine learning models for miRNA expression classification in cervical cancer and Alzheimer's disease.

Article Title: Attentive interpretable tabular learning (TabNet) for miRNA expression classification in cervical cancer and Alzheimer's disease

Article References: Haznedar, H., Özsert Yiğit, G., & Haznedar, B. (2026). Attentive interpretable tabular learning (TabNet) for miRNA expression classification in cervical cancer and Alzheimer's disease. BMC Bioinformatics. https://doi.org/10.1186/s12859-026-06706-7

Image Credits: AI Generated

DOI: 10.1186/s12859-026-06706-7

Keywords: TabNet, miRNA, RNA-seq, cervical cancer, Alzheimer's disease, machine learning, interpretability, TabPFN, XGBoost, deep learning, bioinformatics, small cohorts