AI Pipeline Turns Brain MRI Saliency Maps into Diagnostic Reports

The opaque nature of deep learning models remains a significant barrier to their clinical adoption in medical imaging, including brain tumour MRI classification. A new study in BioData Mining presents a unified pipeline that links image classification, visual saliency mapping, anatomical localisation and large language model (LLM) text generation, aiming to produce diagnostic narratives that clinicians can actually inspect and question.

The framework, described in a peer-reviewed article published on 14 September 2026, operates in three coupled stages. First, nine convolutional neural network (CNN) architectures were extended with a dual-output hybrid formulation: each model simultaneously optimises a classification head and a segmentation head, encouraging the network to learn spatially richer features rather than relying on global image statistics alone.

Second, the researchers applied visual saliency attribution methods—Grad-CAM, Grad-CAM++ and ScoreCAM—to generate class-discriminative heatmaps highlighting which image regions drove each model’s tumour prediction. These continuous heatmaps were then refined into coarse binary masks through an adaptive percentile thresholding pipeline, converting soft attention signals into discrete regions of interest.

Third, the extracted masks were projected onto the Harvard–Oxford cortical atlas as an illustrative anatomical overlay. The authors emphasise that this overlay offers only an approximate visual reference for the tumour’s cortical neighbourhood, not a precise anatomical diagnosis. The atlas-based findings were then encoded into a structured JSON file that conditions three LLMs—Grok3, Mistral and LLaMA—to generate coherent, radiological-style diagnostic reports.

The evaluation used a dataset of 4,834 contrast-enhanced T1-weighted brain MRI images spanning three tumour classes. Among the nine CNN architectures tested, InceptionResNetV2 achieved the highest classification performance, and among the saliency methods, Grad-CAM++ yielded the best segmentation overlap with the reference regions.

The language models showed distinct strengths. Grok3 led in lexical diversity and coherence in the generated reports, while LLaMA achieved the highest readability score.

The study is framed as a systematic empirical benchmark rather than a clinical validation. The atlas overlay is explicitly described as illustrative, and the pipeline was tested on a benchmark dataset rather than deployed in a hospital setting. The authors position the work as an attempt—rather than a proven result—to produce technically grounded and meaningfully interpretable explanations.

Even so, the approach addresses a recognised barrier.

The article is published open access, and the version shared at publication is citable with a permanent DOI, subject to further edits before the final Version of Record. For researchers working on explainable medical AI, the benchmark offers a template for evaluating not just classification accuracy but the full chain of visual, anatomical and linguistic explanation.

Subject of Research: An explainable AI pipeline that converts brain MRI saliency maps into anatomically referenced diagnostic reports using large language models.

Article Title: From visual saliency to anatomical explanation: generating textual diagnostic reports from brain MRIs with saliency-grounded LLMs

Article References: Nguezet, P. V., Fute, E. T., Brima, Y., Azanguezet, B. M., & Atemkeng, M. (2026). From visual saliency to anatomical explanation: generating textual diagnostic reports from brain MRIs with saliency-grounded LLMs. BioData Mining. https://doi.org/10.1186/s13040-026-00601-w

Image Credits: AI Generated

DOI: 10.1186/s13040-026-00601-w

Keywords: brain tumour, MRI, deep learning, explainable AI, saliency maps, large language models, Grad-CAM, Harvard–Oxford atlas, medical imaging, CNN classification, visual, saliency