Knowledge-Distilled Lightweight AI Reads Breast Ultrasounds With Radiologist-Beating Accuracy

A new artificial intelligence model can distinguish benign from malignant breast nodules on ultrasound images with accuracy that exceeds experienced radiologists in a retrospective comparison, while running on a network small enough to fit comfortably on clinical hardware. On an independent private dataset never used during training, the model reached 92.64 percent accuracy with an area under the ROC curve of 0.956, compared with 83.90 percent accuracy for chief physicians who evaluated the same cases.

Ultrasound is a safe, inexpensive, radiation-free imaging method that is particularly valuable for dense breast tissue, where mammography performs poorly. But ultrasound images suffer from noise, limited resolution, blurred lesion boundaries, and strong dependence on the skill of the operator performing the scan. These limitations make benign-malignant diagnosis subjective, and the challenge is especially acute for BI-RADS category 4 nodules, whose reported malignancy rates vary enormously, from 2 percent to 95 percent, leaving clinicians with substantial diagnostic uncertainty.

Deep learning has made rapid progress on this task, and some published models have reported accuracies at or above the level of experienced radiologists. Yet the field faces a trade-off. Large convolutional networks and transformer architectures can capture rich global context, but they demand heavy computation, large training datasets, and substantial memory, which limits their usefulness at the point of care. The new study attacks this trade-off directly by designing a lightweight student network and then boosting its performance through distillation from a stronger teacher, rather than simply scaling up the model.

The architecture integrates three components alongside the distillation mechanism. One of these components is a module termed dual inverted bottleneck blocks (Dual-IB). An adaptive hierarchical feature fusion module combines local texture information with broader semantic cues across multiple stages of the network. A hybrid loss function, termed LMF, is intended to sharpen the model’s discrimination on hard-to-classify samples. During training, a high-performing teacher network produces soft probability targets that convey not just the correct answer but the relative plausibility of alternatives, so-called dark knowledge, which the student learns alongside the hard ground-truth labels.

The choice of distillation hyperparameters proved consequential. Using five-fold cross-validation, the researchers systematically varied the temperature parameter T, which controls how smoothly the teacher’s output distribution is softened, and the distillation weight alpha, which balances soft-target supervision against hard-label supervision. The configuration T = 6 and alpha = 0.5 consistently performed best on both public benchmark datasets, BUSI and UDIAT. Temperatures that were too low carried little dark knowledge, while excessively high temperatures over-smoothed the distributions and weakened the student’s discriminative power. The team also compared four candidate teachers and found that DenseNet121, whose dense connectivity promotes feature reuse, transferred knowledge more effectively than ResNet50, EfficientNet-B0, or VGG16, and it was adopted for all subsequent experiments.

An ablation study confirmed that each component contributed measurably. Starting from a baseline that reached 92.58 percent accuracy on BUSI and 94.36 percent on UDIAT, progressively adding the dual inverted bottleneck blocks, the hybrid loss, the hierarchical fusion module, and finally knowledge distillation lifted the complete model to 95.07 percent accuracy on BUSI and 97.59 percent on UDIAT, gains of 2.49 and 3.23 percentage points over the respective baselines. In head-to-head comparisons under the same five-fold cross-validation protocol, the full model outperformed DarkNet, ShuffleNetV2, ResNet18, MobileNetV3 and V4, Swin-Transformer, ViT, and ConvNeXt on both datasets. Against the strongest baseline, ConvNeXt, the improvement on UDIAT amounted to 1.32 percentage points in accuracy, 1.42 points in F1-score, and 0.006 in AUC.

Because benchmark performance can flatter models that overfit public datasets, the team carried out external validation on an independent private dataset of 503 images that played no role in training. The model again led the field, achieving 92.64 percent accuracy, 92.37 percent F1-score, and an AUC of 0.956, with the strongest baseline, Swin-Transformer, reaching an AUC of 0.952. The researchers also compared the model against six radiologists, two residents, two attending physicians, and two chief physicians, reading the same cases. Accuracy rose with experience, from 68.39 percent for residents to 76.14 percent for attending physicians and 83.90 percent for chief physicians, but the model’s 92.64 percent exceeded all of them. The authors note that the model is intended as an auxiliary tool to support, not replace, radiologists, particularly for less experienced clinicians.

The subgroup analysis reveals where the model still struggles, and the authors are candid about it. On lesions measuring 10 millimeters or smaller, accuracy fell to 87.14 percent, and on low-contrast images, defined by experienced breast ultrasound physicians according to lesion-to-background visibility, it dropped to 88.10 percent. The combination of small size and low contrast was the hardest case of all, with accuracy of 80.65 percent and an F1-score of 79.31 percent. By contrast, lesions larger than 10 millimeters and medium- to high-contrast images yielded accuracies of 93.53 percent and 94.93 percent respectively. Misclassified cases were associated with extremely small lesions, weak contrast, indistinct boundaries, heterogeneous internal echogenicity, and artifacts such as speckle noise and posterior acoustic shadowing, which the authors attribute to the intrinsic difficulty of breast ultrasound imaging rather than flaws in the network design.

Efficiency figures suggest the approach is realistic for deployment. The model requires only 4.12 million parameters and 0.32 gigaflops of computation, with an inference latency of 7.38 milliseconds and peak memory consumption of 23.93 megabytes, all measured under the same hardware and software environment as the baselines. For comparison, ViT used 86.57 million parameters and 612.35 megabytes of memory, while ConvNeXt needed 50.22 million parameters and 398.71 megabytes. Some lighter models, such as ShuffleNetV2 and MobileNetV3, were cheaper still, but they classified images less accurately. Visualization analyses using multiple class activation mapping methods, including GradCAM, GradCAM++, XGradCAM, EigenGradCAM, and LayerCAM, showed that the model’s attention concentrated on lesion regions rather than background tissue, with highlighted areas corresponding to clinically meaningful features such as lesion boundaries, irregular shapes, non-circumscribed margins, and heterogeneous echogenicity.

Several caveats temper the clinical implications. The evaluation was retrospective, based on two public datasets and one private dataset, so cross-device adaptability and performance in real-world clinical workflows remain to be demonstrated, and integration into commercial ultrasound systems has not yet been investigated. The datasets lacked standardized BI-RADS annotations, preventing subgroup analysis of the most diagnostically challenging categories such as BI-RADS 4 nodules, and the correspondence between the model’s attention maps and BI-RADS-related features remains qualitative. The authors outline next steps that include multi-center validation across diverse patient populations and imaging devices, testing on datasets with standardized BI-RADS labels, integrating multi-modal information from mammography or MRI, and exploring few-shot learning to improve detection of small or subtle lesions. The code has been released publicly on GitHub, and the team suggests that the model’s lightweight profile makes it well suited to GPU-equipped clinical workstations for real-time computer-aided diagnosis once prospective validation is complete.

Subject of Research: A lightweight knowledge-distilled deep learning model for diagnosing malignancy in breast ultrasound images

Article Title: Ultrasound-based breast nodule malignancy diagnosis using CNN enhanced by knowledge distillation

Article References: Hu, T., Ru, R., Ding, Y., Wang, V. Y., & Xu, D. (2026). Ultrasound-based breast nodule malignancy diagnosis using CNN enhanced by knowledge distillation. iScience, 29(11), Article 118030. https://doi.org/10.1016/j.isci.2026.118030

Image Credits: AI Generated

DOI: 10.1016/j.isci.2026.118030

Keywords: breast cancer, ultrasound imaging, knowledge distillation, convolutional neural network, computer-aided diagnosis, deep learning, BI-RADS, medical imaging AI, model compression, radiology, BUSI dataset, interpretability