AI Learns to Say I Don’t Know: Risk-Calibrated Transformer Brings Safer Alzheimer’s Staging

Artificial intelligence has promised to transform the diagnosis of Alzheimer’s disease for years, but a persistent problem has kept most models out of the clinic: they are confidently wrong far too often. Now, researchers A. Nasreen Ali and K. P. Madhu, writing in Discover Artificial Intelligence, have unveiled a framework that tackles this weakness head-on. Their model, called RC-TabFormer, does something unusual for a machine learning system — it knows when to abstain. By combining a tabular Transformer with probability calibration, selective prediction, and an ensemble with a gradient-boosted decision tree, the framework delivers not just high accuracy but honest, risk-aware confidence scores that clinicians could actually act on.

The motivation is rooted in a practical dilemma. Most deep learning systems for Alzheimer’s disease rely on neuroimaging — magnetic resonance imaging or positron emission tomography — which can detect patterns of neurodegeneration with impressive precision but is expensive, time-consuming, and unavailable in many routine clinical settings. Tabular clinical biomarkers, by contrast, are collected everywhere: cognitive test scores, demographic information, blood-based markers, genetic data, and cerebrospinal fluid measurements. These data are cheap, scalable, and already sitting in electronic health records. The catch is that they are heterogeneous, messy, and hard for conventional models to weave into a coherent picture of disease stage.

RC-TabFormer’s core innovation lies in how it treats this tabular data. Rather than feeding raw numbers into a network, the model tokenises every feature — each biomarker becomes its own embedding in a shared dimensional space, much like words in a sentence. Numerical values are linearly projected, while categorical variables are one-hot encoded and embedded. A special classification token is prepended to the sequence, and the whole set passes through stacked Transformer encoder blocks built on multi-head self-attention. This mechanism lets the model learn interactions between, say, a dementia rating score and a tau protein ratio, without any hand-crafted rules about which features should talk to each other.

But architecture alone was not the point. The authors identified three chronic gaps in the existing literature: models are rarely probability-calibrated, so a stated 90 percent confidence may be right only 70 percent of the time; models are forced to predict on every patient, even when inputs are ambiguous; and black-box decisions leave clinicians unable to justify or challenge outputs. RC-TabFormer addresses all three. Temperature scaling adjusts the model’s logits so predicted probabilities align with observed outcomes, measured by Expected Calibration Error. A selective prediction module then applies a confidence threshold: cases falling below it are not classified at all but routed to specialist review.

The threshold itself is chosen with statistical rigor. Using an upper confidence bound derived from a beta distribution, the system selects the threshold that maximises coverage — the proportion of patients automatically classified — while guaranteeing that the empirical risk on accepted cases stays below a prespecified bound with high probability. In other words, a hospital could instruct the system to classify as many patients as possible while keeping the error rate among accepted predictions under, say, 6 percent, and the framework would find the operating point that satisfies that contract. This turns an opaque classifier into something closer to a governed clinical instrument.

The performance numbers are striking. On the binary task of distinguishing Alzheimer’s disease from non-AD using a dataset of 2,149 records, RC-TabFormer achieved 95.2 percent accuracy, an F1 score of 96.7 percent, an area under the ROC curve of 98.4 percent, and a Matthews correlation coefficient of 91.2 percent — all under strict subject-level cross-validation that prevents any patient’s data from leaking between training and test sets. Expected Calibration Error was just 1.2 percent, meaning the model’s stated confidence closely tracks its true reliability. On the harder three-class problem of staging patients as cognitively normal, mild cognitive impairment, or Alzheimer’s disease using ADNI longitudinal data, the model reached 89.8 percent accuracy and 95.1 percent macro-AUC, surpassing strong baselines by up to five points.

Ablation studies reveal which components matter most. Removing the ensemble fusion with LightGBM caused the largest performance drop, confirming that tree-based and attention-based learners capture genuinely complementary patterns in the data. Switching off temperature scaling barely changed accuracy but dramatically worsened calibration error — a reminder that a model can be accurate and still dangerously misleading. Disabling selective prediction cost around 1.6 to 1.7 percentage points of accuracy, showing that the willingness to abstain actually sharpens the reliability of the predictions that remain. Coverage-risk curves demonstrated that at 90.2 percent coverage, selective accuracy rose to 94 percent, with mild cognitive impairment cases — the most diagnostically ambiguous group — rejected most often.

Transparency was engineered in from the start. The authors applied SHAP and Integrated Gradients to explain individual predictions, and the two methods agreed strongly, with a mean Spearman rank correlation of 0.78 and a median top-five feature overlap of 0.67. Global rankings were stable across validation folds, and the features the model leaned on were clinically sensible: the Clinical Dementia Rating Scale Sum of Boxes score, ADAS-13 cognitive scores, and ratios of phosphorylated tau to total tau and to amyloid-beta emerged as the dominant drivers of classification. In an external test on an independent NACC cohort, the frozen pipeline retained 86.7 percent balanced accuracy and 92.8 percent AUC, with calibration error of 3.4 percent — a moderate but expected degradation under cohort shift that supports further prospective evaluation.

The authors are careful to position RC-TabFormer as a triage-support system rather than an autonomous diagnostician. Accepted predictions arrive with calibrated probabilities and feature-level explanations for physician review; abstained cases trigger additional cognitive testing, biomarker assessment, or imaging. The abstention threshold is meant to be set by each deploying institution according to its own tolerance for residual risk, disease prevalence, and review capacity. The researchers acknowledge that broader multi-site cohorts, fairness analyses across demographic subgroups, and prospective trials remain necessary before clinical deployment. Still, the message is clear: the next generation of medical AI may earn clinicians’ trust not by always answering, but by knowing exactly when it shouldn’t.

Subject of Research: Risk-calibrated selective tabular Transformer models for explainable Alzheimer's disease staging from clinical biomarkers

Article Title: Risk calibrated selective tabular transformers for explainable Alzheimer’s disease staging

Article References: Ali, A. N., & Madhu, K. P. (2026). Risk calibrated selective tabular transformers for explainable Alzheimer’s disease staging. Discover Artificial Intelligence, 6(1), Article 1349. https://doi.org/10.1007/s44163-026-02361-2

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02361-2

Keywords: Alzheimer's disease, tabular Transformer, selective prediction, probability calibration, explainable AI, SHAP, Integrated Gradients, LightGBM, clinical biomarkers, ADNI, machine learning, clinical decision support