AI Learns to Weigh Its MRI Evidence to Track Multiple Sclerosis Lesions

Multiple sclerosis is a disease of accumulated evidence. Radiologists count and measure the bright white matter lesions that scar the central nervous system, and those measurements guide diagnosis, treatment decisions, and clinical trials. But the deep learning models built to automate this task have a hidden fragility: they expect a complete set of MRI sequences, and real clinical practice rarely delivers one. A team at The Chinese University of Hong Kong has now built a segmentation framework that not only tolerates missing MRI sequences but explicitly weighs the evidence of whatever sequences remain, reporting its results in Frontiers in Radiology.

The problem is more common than most people realize. A standard multisequence MRI protocol for multiple sclerosis typically includes T1-weighted, T2-weighted, fluid-attenuated inversion recovery (FLAIR), proton density, and contrast-enhanced T1-weighted images. Each sequence highlights different tissue properties, and together they give automated systems the complementary contrast needed to distinguish lesions from healthy brain. Yet sequences go missing for mundane reasons: protocol variation between centers, patient motion, image degradation, or scanner downtime. When even one sequence is absent, models trained on complete inputs can stumble badly, because the test-time input distribution no longer matches what they learned during training.

Seung Hoon Lee, Lun Matthew Wong, and Tiffany Y. So tackled this by rethinking how a neural network should represent and combine information from multiple imaging sequences. Their framework, called MG-PoE for multi-scale Gaussian product-of-experts, builds on nnU-Net, a widely respected self-configuring segmentation architecture. Rather than feeding all sequences into a single encoder from the start, MG-PoE processes each sequence separately through a shared encoder, preserving sequence-specific representations until the moment of fusion. At each encoder scale, those representations are parameterized as Gaussian experts, each with its own estimated mean and variance, and then combined through a precision-weighted product-of-experts rule.

The product-of-experts formulation, originally proposed by Geoffrey Hinton in 1999, offers something that learned gating or routing mechanisms do not: a fusion rule that remains mathematically consistent across any observed subset of sequences. Precision is the reciprocal of variance, so sequences associated with lower predicted uncertainty contribute more strongly to the fused representation, while noisier or less informative sequences contribute less. Because the fusion rule is the same regardless of which sequences are present, unavailable experts are simply excluded from the product automatically, without requiring separate architectures for different sequence configurations.

Training under simulated missing-sequence conditions was central to the approach. The researchers trained models in two phases: first with complete multisequence inputs to stabilize sequence-wise representation learning, then under randomly simulated missing-sequence conditions that exposed the network to a broad range of sequence-availability patterns. They also added a weak auxiliary regularizer that discourages fusion-weight collapse onto a single available sequence, softly stabilizing training without strictly enforcing equal weighting. The design keeps the trainable parameter count comparable to the underlying nnU-Net, since encoder weights are shared across sequences.

The evaluation was systematic. On MSSEG2016, a five-sequence benchmark with 53 patients acquired across multiple centers and scanner vendors, MG-PoE achieved the best overall performance across all 26 sequence configurations tested, with a mean Dice Similarity Coefficient of 0.7029, a mean 95th percentile Hausdorff distance of 22.40, and a mean Composite Score of 0.7235. The Dice coefficient quantifies volumetric overlap between predicted and reference segmentations, while the Hausdorff distance assesses boundary agreement, and the Composite Score incorporates lesion detection, false-positive control, and correlation between predicted and reference masks.

On MS3SEG, a three-sequence dataset of 100 patients scanned on a 1.5T Toshiba scanner, MG-PoE achieved the highest mean Dice and Composite Scores across both full-sequence and missing-sequence settings, with significant improvements over competing methods including HeMIS, ModDrop++, and TC-MoE. The Hausdorff distance performance was generally comparable across methods. The ablation study revealed that the benefit arose from combining sequence-wise processing, shared encoder parameterization, and precision-weighted fusion, rather than from any single component alone.

Perhaps most striking was the zero-shot evaluation on MSLesSeg24, a dataset the models had never seen during training. Without any fine-tuning or adaptation, MG-PoE achieved the best overall Dice and Hausdorff performance in the full-sequence setting, with a Dice of 0.5566 and a Hausdorff distance of 27.37, and the best mean Dice, Hausdorff, and Composite Scores across the missing-sequence settings. Absolute performance was lower than on the training datasets, reflecting domain shift between cohorts and acquisition protocols, but the framework maintained competitive accuracy without seeing the target data, providing initial evidence of cross-dataset robustness.

The clinical implications are substantial. Accurate lesion segmentation supports objective radiological assessment, disease monitoring, and treatment response evaluation, and automated methods that deliver consistent measurements under incomplete acquisition protocols could extend the reach of quantitative imaging to centers with limited scanner availability. The framework focuses on configurations with at least two available sequences, since single-sequence settings do not involve cross-sequence fusion, and the authors note that future work should validate the approach in larger multicenter cohorts covering a wider range of scanners and acquisition protocols.

What makes this work resonate beyond multiple sclerosis is the underlying idea: that a neural network should not just process multiple inputs but should represent its confidence in each one and combine them probabilistically. The Gaussian product-of-experts formulation provides a principled way to do this, weighting each sequence’s contribution according to its estimated precision rather than treating all available information equally. As medical imaging becomes increasingly multimodal and increasingly heterogeneous in what data is actually available, frameworks like MG-PoE point toward a future where AI systems can make the most of whatever evidence they are given, and know when to trust it.

Subject of Research: Multi-scale Gaussian product-of-experts for missing sequence multiple sclerosis lesion segmentation

Article Title: Multi-scale Gaussian product-of-experts for missing sequence multiple sclerosis lesion segmentation

Article References: Lee, S. H., Wong, L. M., & So, T. Y. (2026). Multi-scale Gaussian product-of-experts for missing sequence multiple sclerosis lesion segmentation. Frontiers in Radiology, 6, Article 1868811. https://doi.org/10.3389/fradi.2026.1868811

Image Credits: AI Generated

DOI: 10.3389/fradi.2026.1868811

Keywords: Multi-scale, Gaussian, product-of-experts, missing, sequence, multiple, sclerosis, lesion, segmentation, scientific research