Attention-Guided Distillation Boost Alzheimer’s Stage Identification Using MRI Scans

A novel attention–guided cross-representation distillation framework has been developed to significantly enhance the identification of Alzheimer’s disease (AD) stages using Magnetic Resonance Imaging (MRI) scans, specifically targeting subtle and spatially distributed anatomical variations across axial, coronal, and sagittal views. This research addresses critical limitations in existing attention-based deep learning approaches, which often struggle to accurately capture the nuanced anatomical changes characteristic of AD progression, such as hippocampal atrophy, ventricular enlargement, and cortical thinning. By integrating advanced transformer architectures and a unique distillation mechanism, the proposed framework achieves classification accuracies of up to 98.8% on benchmark datasets like ADNI and AIBL, while simultaneously providing robust interpretability through sophisticated visualization techniques.

The core challenge in diagnosing AD progression lies in the subtle nature of the anatomical deviations that occur over time. Traditional attention-based models frequently rely on either global or local attention representations. This reliance limits their capacity to jointly capture long-range anatomical dependencies—the way changes in one brain region might correlate with subtle abnormalities in a distant, interconnected area—and to pinpoint specific, subtle region-specific abnormalities that define the progression of the disease. The proposed solution overcomes these limitations by proposing an attention–guided cross-representation distillation framework that strategically integrates two powerful transformer models: the Vision Transformer (ViT) and the Swin Transformer (SWT).

The architectural innovation centers on a gated dual-teacher single-student design. Within this structure, the teacher network is formed using Interfusion Attention Fusion (IFA). IFA is a mechanism designed to fuse attention patterns across different representations, allowing the teacher to synthesize both global context and local detail. The student model, which is a lightweight ViT-Lite, is then trained through an online response-based distillation mechanism. This distillation process is crucial; it enables the student model to effectively replicate the complex spatial attention patterns learned by the powerful teacher network, thereby inheriting the ability to recognize discriminative anatomical cues essential for accurate AD stage identification without needing to replicate the teacher’s full computational complexity.

The integration of ViT and SwinT within this dual-teacher setup allows the system to capture both global and local representations of the MRI data. The ViT component excels at capturing long-range dependencies across the entire image, providing a holistic view of the brain structure, while the SwinT component excels at capturing fine-grained, local representations. The IFA mechanism then fuses these two distinct types of attention, creating a highly comprehensive teacher model capable of identifying the subtle, spatially distributed anatomical variations that define AD progression across different MRI views (axial, coronal, and sagittal). This dual-perspective approach is critical for robust diagnosis.

Experimental validation of the framework was conducted using two established benchmark datasets: ADNI and AIBL. The results demonstrated exceptional performance, achieving classification accuracies reaching up to 98.8%. Beyond predictive performance, the study reported interpretability results, including Grad-CAM visualizations, LIME analyses, and Bland–Altman agreement between teacher and student predictions. Grad-CAM visualizations, when paired with an Attention Consistency Score (ACS) of up to 96%, effectively demonstrated the successful transfer of spatial reasoning from the teacher to the student network. This high ACS confirms that the student model is not merely mimicking outputs but is genuinely learning and replicating the critical spatial attention patterns emphasized by the teacher.

Further confirmation of the model’s reliability and anatomical relevance came from applying Local Interpretable Model-agnostic Explanations (LIME). LIME analysis yielded Intersection over Union (IoU) scores of up to 0.61 and Dice similarity coefficients of up to 0.72. These metrics confirm a strong agreement in the anatomically relevant regions that are emphasized by both the teacher and the student networks. Furthermore, the use of Bland–Altman analysis provided quantitative verification of strong probabilistic agreement between the predictions generated by the teacher and the student models, reinforcing the stability and reliability of the distillation process.

The robustness of the proposed framework was further confirmed through detailed ablation and comparative studies. These analyses highlight the framework’s unique ability to achieve both high predictive performance and clinically meaningful interpretability simultaneously. By distilling the spatial reasoning of the dual-teacher model into a lightweight student model, the framework achieved high predictive performance while remaining computationally lightweight.

In summary, this research introduces a sophisticated deep learning solution that moves beyond simple classification. By employing an attention–guided cross-representation distillation framework, it tackles the complex task of identifying subtle AD progression markers in MRI scans. The combination of ViT and SwinT, fused via IFA, and then distilled into a lightweight student model, represents a significant advancement in medical image analysis, offering high accuracy (up to 98.8%) coupled with strong, verifiable anatomical interpretability (ACS up to 96%). The framework achieved classification accuracies of up to 98.8% on the ADNI and AIBL benchmark datasets, with interpretability metrics including an Attention Consistency Score of up to 96%.

Subject of Research: Development of a deep learning framework combining Vision Transformer and Swin Transformer for accurate Alzheimer's stage identification from MRI scans.

Article Title: Attention–guided cross-representation distillation for Alzheimer’s stage identification employing MRI scans

Article References: Barman, C., & Barma, S. (2026). Attention–guided cross-representation distillation for Alzheimer’s stage identification employing MRI scans. Neural Computing and Applications, 38(20), Article 848. https://doi.org/10.1007/s00521-026-12568-0

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12568-0

Keywords: Alzheimer's Disease, MRI, Deep Learning, Vision Transformer, Swin Transformer, Medical Imaging, Attention Mechanisms, AI, Cross-Representation Distillation, Attention, guided, cross-representation