Multimodal AI Classifies Herpes Simplex Keratitis Subtypes Across Three Eye Centers

Herpes simplex keratitis, a corneal infection caused by herpes simplex virus type 1, remains one of the leading causes of infectious blindness worldwide, with an estimated global incidence of roughly 1.5 million cases and about 40,000 new cases of severe monocular visual impairment or blindness each year. A team of researchers in China now reports a multimodal deep learning system, called DeepFusion-HSK, that can distinguish between the two therapeutically critical forms of the disease with accuracy levels that outperformed several established artificial intelligence architectures. The work, published in Frontiers in Cell and Developmental Biology, was validated across three independent clinical centers and demonstrated a degree of resilience to missing imaging data that its developers say is essential for real-world clinical deployment.

The clinical stakes of the classification task are unusually high. Epithelial herpes simplex keratitis, the most common recurrent form, presents as characteristic corneal ulcers and demands immediate, aggressive antiviral therapy. Non-epithelial disease, involving viral endotheliitis or deep stromal inflammation, instead requires careful immunomodulation with topical corticosteroids alongside antiviral prophylaxis. Confusing the two can be catastrophic: if epithelial disease is mistaken for non-epithelial inflammation and treated with corticosteroids, the drugs can trigger rapid viral replication, ulcer progression, and severe visual loss. In many developing regions without access to polymerase chain reaction testing, diagnosis rests entirely on slit-lamp biomicroscopy, and many cases are misclassified as unexplained corneal opacities or non-infectious keratitis.

Despite the transformation that artificial intelligence has brought to ophthalmic image analysis, from diabetic retinopathy screening to glaucoma detection, most existing systems for keratitis rely on a single imaging modality, typically diffuse slit-lamp photographs. Moreover, the researchers note that nearly all prior studies have lumped epithelial and non-epithelial herpes simplex keratitis into a single viral keratitis category, overlooking the therapeutic distinction that separates them. DeepFusion-HSK was designed specifically to close this gap, building on the team’s earlier work on multimodal infectious keratitis classification.

The study was retrospective and multicenter. The internal development set comprised 3,501 slit-lamp images collected from 807 eyes at the Eye Hospital of Wenzhou Medical University between 2018 and 2025, divided into training, validation, and test sets in a 7:1.5:1.5 ratio with stratified sampling at the patient level to prevent data leakage. Two external cohorts provided independent validation: External Dataset I contained 1,780 images from 142 patients at Aier Guangming Eye Hospital in Ningbo, and External Dataset II contained 1,177 images from 131 patients at the Second Affiliated Hospital of Zhejiang University School of Medicine in Hangzhou. Reference diagnoses were drawn from final documented clinical records, verified jointly by two ophthalmologists, with cases of uncertain diagnosis, mixed infection, or incomplete information excluded.

The technical core of the framework is its fusion of three complementary illumination modalities. Diffuse light imaging provides uniform anterior-segment illumination for assessing overall surface morphology; slit-beam imaging uses a narrow optical section to reveal corneal depth, contour, and thickness; and fluorescein staining imaging, under cobalt-blue light, highlights epithelial defects as green-stained areas. Each modality is processed by its own DenseNet121 backbone, initialized with weights from a model previously trained on large-scale, multicenter ophthalmic data. The three 1024-dimensional feature vectors are combined through a tri-way higher-order interaction layer, in which they are multiplied element-wise to capture synergistic cross-modal pathological patterns, then concatenated with the original modality-specific vectors into a 4096-dimensional representation.

Several additional design choices target known weaknesses of conventional classifiers. A dual-task architecture routes the fused features to two parallel heads: the first distinguishes viral from non-viral infection to provide macro-level prior knowledge, while the second performs the fine-grained three-class classification of epithelial disease, non-epithelial disease, and other corneal abnormalities. Independent auxiliary heads attached to each backbone prevent any single modality’s features from degrading during training, and a cost-sensitive loss assigns higher weight to the epithelial class to counter class imbalance. At test time, a dynamic gating mechanism uses high-confidence output from the first classifier as a prior to calibrate the second, mitigating the probability spillover artifacts common in standard softmax outputs. Training used sharpness-aware minimization with AdamW, cosine-annealed learning rates, and exponential moving averaging of weights.

On the internal test set, DeepFusion-HSK achieved an accuracy of 85.1 percent, with 95 percent confidence intervals of 78.5 to 90.0 percent, and area under the receiver operating characteristic curve values of 0.947, 0.949, and 0.940 for epithelial disease, non-epithelial disease, and other corneal abnormalities respectively. It clearly outperformed Swin-Transformer at 81.1 percent accuracy, EfficientNetV2 at 77.6 percent, and ResNet50 at 57.1 percent, with corresponding F1 scores of 84.1, 78.5, 75.1, and 46.4 percent. External validation confirmed robust generalization: accuracy was 81.7 percent on Dataset I, and 78.2 percent on Dataset II, where a notable challenge arose because the fluorescein staining modality was entirely absent; the researchers fine-tuned the model on images from 30 randomly selected patients before testing on the remaining 101, and the system retained its lead over all comparison algorithms.

Ablation experiments quantified how much each modality contributes. The fluorescein-based model alone achieved the best single-modality accuracy at 78.5 percent, reflecting its power to expose dendritic ulcers and epithelial defects; diffuse light imaging alone reached 73.5 percent; and slit-beam imaging alone lagged at 63.6 percent, likely because a narrow optical section provides depth information but lacks global morphology and carries a lower effective signal-to-noise ratio. Among dual-modality combinations, diffuse light plus fluorescein staining reached 81.7 percent, and full tri-modal fusion lifted performance to 85.1 percent. The hierarchy mirrors clinical reasoning, in which a physician first inspects global morphology, then probes depth, then checks for superficial ulceration. Grad-CAM heatmaps further showed that the network concentrated on clinically relevant lesion regions rather than background artifacts.

The authors argue that the fusion strategy, which processes each modality independently before integrating at the feature level, is key to deployability. Early-fusion architectures that require complete modality alignment degrade sharply when fluorescein is omitted or a slit-beam capture is skipped in a busy clinic. In contrast, the missing-modality test on Dataset II showed only a modest drop from 81.7 to 78.2 percent after light fine-tuning, a graceful degradation rather than complete failure. The researchers position this as a decision-support tool, particularly for early-career ophthalmologists in resource-limited settings, intended to reduce vision-threatening misdiagnoses and guide therapy.

The study carries acknowledged limitations. Its retrospective design is subject to selection bias, and the reference standard relied on clinical records rather than prospective molecular confirmation, since herpes simplex keratitis is primarily diagnosed clinically and polymerase chain reaction sensitivity varies across subtypes; the authors call for future studies incorporating standardized imaging with laboratory testing of corneal scrapings or tear samples. External validation was also limited to two Chinese centers, leaving performance in other geographic and healthcare contexts unknown. Even so, the researchers conclude that DeepFusion-HSK demonstrates that task-specific, interpretable multimodal systems can deliver accurate, therapeutically actionable subtype classification across diverse ophthalmic settings, even when ideal imaging inputs are unavailable.

Subject of Research: Multimodal deep learning for subtype classification of herpes simplex keratitis using slit-lamp imaging with multicenter validation

Article Title: DeepFusion-HSK: a multimodal deep learning for subtype classification of herpes simplex keratitis with multicenter validation

Article References: Zhang, Z., Lv, T., Tang, J., Xu, K., Li, F.-F., Yu, X., Wang, C., Wu, S., Fu, Y.-N., Jiang, J.-W., Lu, F., & Dai, Q. (2026). DeepFusion-HSK: a multimodal deep learning for subtype classification of herpes simplex keratitis with multicenter validation. Frontiers in Cell and Developmental Biology, 14, Article 1978214. https://doi.org/10.3389/fcell.2026.1978214

Image Credits: AI Generated

DOI: 10.3389/fcell.2026.1978214

Keywords: herpes simplex keratitis, multimodal deep learning, slit-lamp imaging, subtype classification, ophthalmology, corneal disease, artificial intelligence, multicenter validation, fluorescein staining, DenseNet121, medical imaging, decision support