Multi-scale transformer network sharpens brain tumor segmentation on MRI scans

Researchers in China have developed a deep learning architecture that combines multi-scale convolutional feature extraction with sparse transformer attention to segment brain tumors from multimodal magnetic resonance imaging, reporting competitive Dice scores across three clinically defined tumor subregions on two widely used public benchmarks. The network, called MTF-Net, was described in a study published in Frontiers in Medicine on 26 October 2026 by a team from the North University of China in Taiyuan and Inner Mongolia University of Technology in Ordos. On the BraTS2019 dataset, the model achieved average Dice coefficients of 0.926 for the whole tumor, 0.921 for the tumor core, and 0.896 for the enhancing tumor. On the BraTS2021 dataset, the corresponding scores were 0.918, 0.923, and 0.901. The authors report that these values were higher than those of the evaluated comparison methods on most metrics, although not on every single one.

Brain tumors, particularly gliomas, are aggressive malignancies that threaten the central nervous system and impair cognitive and neurological function. Magnetic resonance imaging is the standard non-invasive tool for detecting, classifying, and evaluating these tumors, and the four MRI modalities used in the BraTS benchmarks—native T1-weighted, post-contrast T1-weighted, T2-weighted, and T2 fluid-attenuated inversion recovery sequences—provide complementary tissue contrast that helps clinicians identify tumor regions. Manual segmentation, however, depends on expert experience, is time-consuming, and is subjective, which has driven the adoption of automatic deep learning methods built on fully convolutional networks and the U-Net encoder-decoder design.

Convolutional neural networks excel at local feature extraction, but their limited receptive fields struggle with the heterogeneity, blurry boundaries, and irregular morphology of brain tumors. Transformers, introduced to medical segmentation through architectures such as TransUNet and Swin-Unet, offer strong global modeling, yet they carry high computational costs and heavy memory demands. Conventional attention mechanisms like SE and CBAM can enhance feature expression but suffer from parameter redundancy and weak spatial modeling. The authors argue that existing methods tend to improve multi-scale modeling, transformer attention, and feature fusion in isolation, and that integrating these capabilities remains difficult for heterogeneous lesions.

MTF-Net addresses these gaps through three purpose-built modules arranged hierarchically. The Multi-Scale and Spatially-Aware Transformer module, or MSFormer, operates in the second to fourth encoding layers. It pairs a Multi-Scale Depthwise Convolution block, which runs three parallel depthwise convolution branches at different receptive fields and concatenates their outputs before channel compression and Softmax weighting, with a Spatially-Aware MLP that applies a 3×3 depthwise convolution before pointwise channel projection to preserve spatial structure. This parallel design lets the network model local detail and long-range context simultaneously at stages where spatial resolution is still sufficient for structural analysis.

At the deepest encoding stage, the Channel-Enhanced Bi-level Attention Module, CEBAM, takes over. Built on the Bi-Level Routing Attention mechanism from BiFormer, it first applies a 3×3 depthwise separable convolution to capture local edges and textures, then uses region-level similarity scoring to select the top-k most relevant regions for fine-grained token attention, a sparse strategy that avoids the expense of fully connected attention. A spatially-aware MLP and residual connections then strengthen channel-spatial interactions. The authors state that this combination enhances the expression of long-range dependencies while keeping parameters in check on the reduced-resolution feature map.

Bridging the encoder and decoder, the Cross-Modal Feature Interaction Module, CFIM, replaces the direct concatenation or addition used in typical U-Net-style skip connections. It adaptively fuses the contextual features produced by the MSFormer branch with the local structural features from the parallel convolutional branch, generating content-dependent fusion weights through coordinate attention and Softmax normalization. Two symmetric multi-path convolutional branches, each combining a single and a stacked pair of 3×3 depthwise separable convolutions, refine the two feature streams before weighted summation. According to the authors, this design narrows the semantic gap between encoder and decoder and improves boundary perception and consistency across modalities.

Training details are extensive. The team used the full BraTS2019 cohort of 335 glioma patients, comprising 76 low-grade and 259 high-grade cases, and the 1,251 labeled training cases from the BraTS2021 benchmark, splitting both datasets at the patient level into training, validation, and testing subsets in a 7:1:2 ratio to prevent slice-level data leakage. Three-dimensional volumes of 240×240×155 voxels were center-cropped to 160×160 pixels after mask verification confirmed that all tumor regions fell within the cropped window, then converted into 155 two-dimensional axial slices each. Intensities were clipped at the 1st and 99th percentiles and normalized, and a hybrid loss combining binary cross entropy, weighted at 1.5, with Dice loss addressed the severe class imbalance between small tumor regions and background.

The model was trained for 50 epochs with the Adam optimizer, an initial learning rate of 0.001, and cosine annealing on an NVIDIA RTX 3090, with the best checkpoint selected at epoch 48 based on validation loss. Ablation experiments on BraTS2019, using a TransUNet-based baseline that scored 0.852, 0.834, and 0.810 Dice for whole tumor, tumor core, and enhancing tumor, showed that adding CEBAM, MSFormer, and finally CFIM raised those scores stepwise to 0.926, 0.921, and 0.896 while lowering the 95th-percentile Hausdorff distance to 5.123, 5.310, and 3.203 millimeters. MTF-Net contains 100.28 million parameters and requires 79.30 gigaflops per four-modality input, substantially less computation than several compared 3D and hybrid methods, and full training took roughly 10 hours with about 20 gigabytes of GPU memory.

The authors are careful to frame the results as internal rather than official benchmark outcomes, since the BraTS validation and testing servers were not used, and they note that all baselines were reimplemented under a unified protocol with a fixed random seed of 26. Patient-level Wilcoxon signed-rank tests yielded p-values below 0.05 for all regions on both datasets, but the team emphasizes that single-run results without repeated seeds or confidence intervals should not be interpreted as statistically significant improvements. They also acknowledge that both datasets come from the same benchmark family, that the 2D slice-based approach does not exploit information from adjacent slices, and that external multi-center validation, expert reader evaluation, and prospective clinical assessment will be required before any clinical translation.

Subject of Research: Deep learning segmentation of brain tumors from multimodal MRI using a multi-scale transformer and feature fusion network

Article Title: MTF-Net: a brain tumor segmentation network with multi-scale transformer and feature fusion enhancement

Article References: Liu, W., Cao, Y., Liu, Y., Zhang, P., Gui, Z., & Liu, L. (2026). MTF-Net: a brain tumor segmentation network with multi-scale transformer and feature fusion enhancement. Frontiers in Medicine, 13, Article 1934161. https://doi.org/10.3389/fmed.2026.1934161

Image Credits: AI Generated

DOI: 10.3389/fmed.2026.1934161

Keywords: brain tumor segmentation, MTF-Net, transformer, multimodal MRI, BraTS2019, BraTS2021, deep learning, attention mechanism, feature fusion, Dice coefficient, glioma, medical imaging