Adaptive Decoder Learns Where to Fuse Features in Medical Image Segmentation

Medical image segmentation models often struggle because they combine information from different layers of a neural network in a fixed way. A new study introduces DynamicFusionNet, a decoder framework that instead decides dynamically, for each input, how features from different layers, scales and representational views should be merged. The work, published in the journal Cancer Imaging, was motivated by the observation that standard segmentation architectures rely on static fusion to aggregate multi-level features, an approach that often fails to accommodate the heterogeneity of medical images.

The researchers describe DynamicFusionNet as a drop-in decoder that learns dense fusion policies conditioned on the input. Rather than treating every scan the same way, the network adjusts its internal merging strategy depending on what it sees. This matters in clinical settings, where images vary widely in modality, quality and anatomy, and a single rigid recipe for combining features may suit some cases poorly.

The framework has three main components. First, it applies pixel-wise gating on skip pathways, which allows the model to select, at the level of individual pixels, which layer features are worth passing forward. Second, a multi-scale calibration module integrates complementary branch responses through a shared pointwise projection, so that information captured at different spatial scales is weighted appropriately. Third, a dual-pathway global-local arbitration design balances high-level semantics with fine boundary details, helping the model keep track of both what a region is and where its edges lie.

A key practical feature is that these modules can be integrated into standard encoder-decoder pipelines without modifying the backbone. The authors position DynamicFusionNet as a modality-agnostic fusion principle that can be adopted in existing segmentation systems to improve clinical image analysis without bespoke architectures per task. In other words, teams working with different imaging types could potentially swap in this decoder rather than redesigning their entire networks.

The study reports experiments on five public benchmarks, with main results on BraTS 2021, RETOUCH and PARSE 2022. These datasets cover brain tumour imaging with multiple magnetic resonance modalities, retinal imaging, and cardiac structures, giving the evaluation a spread across multi-modal and single-modal settings. According to the abstract, DynamicFusionNet consistently outperforms strong CNN and Transformer baselines in segmentation accuracy and boundary fidelity across these settings.

Boundary fidelity is a particularly important measure in medical imaging. A segmentation that labels the correct region but draws sloppy edges can mislead measurements of tumour volume or organ shape. The authors attribute the improvement to combining input-adaptive layer and view fusion with learned multi-scale response calibration, which they say provides a unified and effective alternative to static decoder aggregation and improves robustness under heterogeneous imaging conditions.

The paper is published as open access research, and the authors have released code publicly at a GitHub repository, which lowers the barrier for other groups to test the approach on their own data. Because the decoder is designed to be a drop-in component, the intended workflow is to attach it to an existing encoder-decoder model and compare performance before and after, rather than retraining an entirely new architecture from scratch.

It is worth noting the scope of the evidence. The reported gains come from benchmark experiments on public datasets, not from prospective clinical trials, so the results describe performance under research conditions rather than validated diagnostic benefit in routine care. The article is also being shared early as peer-reviewed, accepted research that is citable and carries a permanent DOI, with a note that this version is subject to further edits and will be replaced automatically by the final Version of Record.

Even so, the central idea is straightforward and broadly applicable: let the network learn when and where to fuse, instead of fixing that policy in advance. If the benchmark results hold up as the method is tested more widely, adaptive fusion could become a practical default for medical segmentation systems that must handle the messy variety of real-world clinical images.

Subject of Research: Adaptive multi-level feature fusion for multimodal medical image segmentation

Article Title: Dynamic layer_view_scale fusion for multimodal medical segmentation

Article References: Ding, J., Zhou, M., Zhao, J., Hou, L., Wang, D., Duan, Y., & Zhang, H. (2026). Dynamic layer_view_scale fusion for multimodal medical segmentation. Cancer Imaging. https://doi.org/10.1186/s40644-026-01150-5

Image Credits: AI Generated

DOI: 10.1186/s40644-026-01150-5

Keywords: medical imaging, image segmentation, deep learning, DynamicFusionNet, neural networks, CNN, Transformer, BraTS, RETOUCH, PARSE 2022, decoder architecture, open access