AI Learns to Trust the Right Kind of Skin Photo: Dermoscopy-Dominant Fusion Boosts Lesion Diagnosis

When dermatologists examine a suspicious mole or patch of skin, they rarely rely on a single view. They look at the lesion with the naked eye, capturing its color, size, and context on the body, and then they switch to a dermatoscope, a handheld magnifying device with its own light source that reveals subsurface structures invisible to ordinary photography. Both views matter, but they are not equally informative. A new study published in BMC Medical Imaging by Yusung Chu and Sejung Yang of Yonsei University Wonju College of Medicine and Konyang University takes this clinical intuition seriously and builds it directly into the architecture of an artificial intelligence system designed to classify skin lesions across nine diagnostic categories.

The core insight behind the work is that clinical photographs and dermoscopic images, while complementary, carry fundamentally different kinds of information. Dermoscopic images expose detailed morphological features such as pigment networks, globules, and vascular patterns that are central to differential diagnosis. Clinical images, by contrast, situate the lesion within its broader anatomical and visual context but offer less diagnostic resolution at the cellular and structural level. Most existing multimodal fusion approaches in medical imaging treat both modalities symmetrically, blending their features with equal weight as if neither view were inherently more diagnostic. Chu and Yang argue that this symmetry is a mistake, and they propose an alternative they call Dermoscopy-Dominant Correction Fusion, or DDCF.

DDCF is an asymmetric fusion framework. Rather than merging clinical and dermoscopic features as equals, it preserves the dermoscopic representation as the base, the primary backbone of the model’s understanding of the lesion. Clinical information then enters not as an equal partner but as a gated corrective signal, a mechanism that modulates and refines the dermoscopic features without overwhelming them. This design choice reflects the clinical reality that dermoscopy is the more diagnostically powerful modality for skin lesion classification, and that clinical context should serve to correct or contextualize rather than to dominate. The framework was tested across three different neural network backbones: Swin-Tiny, a vision transformer variant; DenseNet121, a densely connected convolutional network; and EfficientNet-B4, a compound-scaled convolutional architecture. Testing across multiple backbones is important because it helps distinguish improvements attributable to the fusion strategy itself from improvements attributable to any single model’s particular inductive biases.

The evaluation was conducted on the MILK10k dataset, using nine diagnostic categories that span a clinically meaningful range of skin lesions. The researchers created three stratified lesion-level partitions with a 7:1:2 training, validation, and test ratio, holding class-wise counts fixed across splits. This stratification ensures that rare diagnostic categories are not underrepresented in any partition, a critical concern in medical imaging where class imbalance can silently inflate or deflate performance metrics. For each backbone and each partition, the correction coefficient was fixed at a specific value, and the model checkpoint with the lowest validation loss was selected before any test evaluation was performed. This protocol guards against overfitting to the test set and provides a more honest estimate of real-world generalization.

The primary evaluation metric was macro-averaged one-vs-rest area under the receiver operating characteristic curve, or AUROC, which measures the model’s ability to rank positive cases above negative ones across all diagnostic categories regardless of their prevalence. Balanced accuracy and macro-F1 score served as complementary metrics. To compare DDCF against a baseline, the researchers used naive-sum fusion, a straightforward approach that simply adds the feature representations from both modalities without any gating or asymmetry. Statistical comparison was performed using paired stratified bootstrap confidence intervals within each backbone and partition combination, yielding nine total backbone-by-partition comparisons.

The results paint a nuanced picture. At a correction coefficient of 0.7, DDCF achieved mean test AUROCs of 0.9308 for Swin-Tiny, 0.9099 for DenseNet121, and 0.9084 for EfficientNet-B4. This coefficient produced the highest observed mean test AUROC among all evaluated values for all three backbones, suggesting that a moderate but clearly dermoscopy-dominant weighting is optimal. In all nine backbone-by-partition comparisons, the AUROC difference between DDCF and naive-sum fusion was positive, meaning that the asymmetric approach numerically outperformed the symmetric baseline every single time. In two of those nine comparisons, the nominal 95 percent bootstrap confidence intervals excluded zero, providing formal statistical support for the improvement in those specific configurations. Macro-F1 differences were also positive across all nine comparisons, indicating that the gains were not limited to a single metric.

However, the study is careful not to overstate its findings. Balanced accuracy told a more complicated story: for Swin-Tiny, balanced accuracy was actually lower with DDCF than with naive-sum fusion in all three partitions, with a mean of 0.5736 versus 0.6229 for the baseline. This discrepancy between AUROC and balanced accuracy highlights an important tension in medical AI evaluation. AUROC measures ranking ability across thresholds, while balanced accuracy depends on a specific operating point, and a model can improve in one while declining in the other. The researchers also found that class-wise improvements were particularly evident for benign keratosis-like lesions, abbreviated BKL, and basal cell carcinoma, abbreviated BCC. Yet even here, the increased sensitivity for BCC came at the cost of reduced specificity, meaning the model became better at catching BCC cases but also more prone to false positives for that category.

Perhaps most notably, the authors found no uniform benefit for melanoma or for keratinocytic tumors as a whole. This is a significant caveat given that melanoma detection is often the primary motivation for advancing skin lesion classification systems. The results do not establish a universally optimal correction coefficient, nor do they demonstrate general clinical superiority of DDCF over simpler fusion approaches. What they do establish is a consistent directional signal: the asymmetric, dermoscopy-dominant approach produced positive numerical AUROC differences in every tested configuration, which is a meaningful pattern even if individual comparisons did not all reach conventional statistical significance.

The work is funded by the National Research Foundation of Korea through the Ministry of Science and ICT and represents a growing recognition that how multimodal information is combined in medical AI is as important as which modalities are used. Rather than treating fusion as a solved problem, Chu and Yang demonstrate that the architecture of combination itself, specifically which modality serves as the foundation and which serves as a corrective influence, can measurably affect diagnostic performance. The study analyzed a publicly available, de-identified dataset and did not enroll new human participants, and the authors declare no competing interests. As multimodal medical AI continues to mature, findings like these suggest that the field may benefit from moving beyond symmetric fusion paradigms and toward designs that explicitly encode the clinical hierarchy of diagnostic evidence, letting the most informative modality lead while the other provides context and correction.

Subject of Research: Asymmetric multimodal fusion of clinical and dermoscopic images for AI-based skin lesion classification

Article Title: Dermoscopy-dominant correction fusion for paired clinical and dermoscopic skin lesion image classification

Article References: Chu, Y., & Yang, S. (2026). Dermoscopy-dominant correction fusion for paired clinical and dermoscopic skin lesion image classification. BMC Medical Imaging. https://doi.org/10.1186/s12880-026-02955-3

Image Credits: AI Generated

DOI: 10.1186/s12880-026-02955-3

Keywords: dermoscopy, skin lesion classification, multimodal fusion, medical imaging, artificial intelligence, deep learning, clinical images, AUROC, melanoma, basal cell carcinoma, computer vision, diagnostic AI