New Explainability Score Could Catch AI’s Dangerous Breast Cancer Misses

Deep learning models can now classify breast tissue biopsies with near-perfect accuracy, yet many pathologists remain reluctant to trust them. The reason is simple: when a neural network declares a slide malignant or benign, it offers no insight into why. A new study published in Frontiers in Imaging tackles this trust gap head-on, presenting a framework that does not merely explain AI decisions but converts those explanations into a single number that can predict when the model is about to be wrong.

The research, led by Raimondo Fanale and colleagues at Universitas Mercatorum in Rome, working with the National Interuniversity Consortium for Telecommunications, focuses on histopathology, the microscopic examination of stained tissue sections that remains the gold standard for cancer diagnosis. The team built their system on the BreaKHis dataset, a public collection of 7,909 breast tissue micrographs from 82 patients, spanning four magnification levels and eight tumor subtypes, from benign adenosis to malignant ductal carcinoma.

At the heart of the pipeline is a knowledge-distilled convolutional neural network. An InceptionV3 model, prized for its multi-scale feature extraction, served as the teacher, while a DenseNet-121 student learned to mimic its outputs. Distillation compresses the teacher’s knowledge into a more compact student, and the payoff was striking: the student achieved a training-to-validation accuracy gap of less than 0.002, indicating negligible overfitting, while surpassing its own teacher on every test metric.

On the 1,187-image held-out test set, the distilled student reached an accuracy of 0.9916, an F1-score of 0.9939, and an area under the receiver-operating-characteristic curve of 0.9989, with sensitivity of 0.9939 and specificity of 0.9866. Five-fold cross-validation using the dataset’s official patient-level folds yielded a mean accuracy of 0.8962 and a mean AUC of 0.9598, figures the authors describe as the more conservative and representative estimate of generalization, since the single test cohort of 13 patients represents a favorable draw.

But raw accuracy is only half the story. The framework’s real innovation lies in how it interrogates the model’s reasoning. For every image, three complementary explanation techniques generate heatmaps: Grad-CAM traces gradients through the final convolutional layers to localize diagnostically relevant regions; LIME perturbs superpixels and fits a local surrogate model to measure each region’s influence; and SHAP, grounded in cooperative game theory, assigns attribution scores quantifying how much each part of the image pushes the prediction toward malignancy or benignity.

These three maps are then fused into the ExpiScore, a composite interpretability index with formal mathematical properties including boundedness, monotonicity, and linearity in its balancing parameter. The score combines a mean stability term, reflecting the average quality of the three individual explanations, with an OGSL consensus term that measures overlap, geometric coherence, and structural similarity across methods. Higher ExpiScore values signal explanations that are not only strong but mutually consistent, a property the authors argue is essential for clinical credibility.

The results reveal a striking pattern. ExpiScore is highest for true positives at 0.313, followed by true negatives at 0.279, false positives at 0.258, and false negatives at just 0.224. Correctly identified malignant cases, the most clinically critical category, consistently exhibit the most coherent explanations, while missed cancers show the weakest. A Spearman correlation of 0.674 between ExpiScore and model confidence confirms the two signals are related but far from redundant, meaning explanations carry information that confidence alone cannot capture.

This insight powers the framework’s most consequential component: a trust meta-model built from logistic regression and a decision tree, trained on 42 features derived from the explanation maps. The meta-model achieved an AUC of 0.9030 in distinguishing correct from incorrect predictions on held-out data. When deployed as a routing mechanism, it flags the 20 percent of cases with the highest predicted error probability for expert review, raising post-routing accuracy from 0.9898 to 0.9983 and cutting false negatives by 83.3 percent, under the assumption that experts correctly resolve all flagged cases.

The authors are careful to keep two operational modes strictly separate. In fully automatic mode, the CNN alone achieves the headline figures without any human involvement. In human-in-the-loop mode, the meta-model triages cases into three categories: accept for high-confidence, coherent predictions; review for ambiguous ones; and defer for cases exhibiting both low confidence and low explanatory coherence. The improvement in the second mode is decisional rather than algorithmic, as the CNN’s predictions remain unchanged while the system becomes better at identifying which ones deserve scrutiny.

Sensitivity analyses confirm the approach is robust: the balancing parameter performs at least as well at its default value as at any alternative, and even if experts resolve only 70 percent of routed cases correctly, system accuracy still exceeds the CNN-only baseline. The authors acknowledge limitations, including the computational cost of roughly 25 to 45 seconds per image for full explanation generation, the single-institution dataset, and the need for external multi-cohort validation before clinical deployment. Yet the core message stands: explainability is not merely a transparency accessory but an operational instrument that can catch the errors confidence alone conceals.

Subject of Research: A quantitative explainability framework for deep learning-based breast cancer histopathology classification

Article Title: A quantitative framework for explainability assessment in medical imaging

Article References: Fanale, R., Caldelli, R., Martini, B., & Sciarrone, F. (2026). A quantitative framework for explainability assessment in medical imaging. Frontiers in Imaging, 5, Article 1846414. https://doi.org/10.3389/fimag.2026.1846414

Image Credits: AI Generated

DOI: 10.3389/fimag.2026.1846414

Keywords: explainable AI, histopathology, breast cancer, Grad-CAM, SHAP, LIME, knowledge distillation, medical imaging, deep learning, ExpiScore, clinical decision support, false negatives