Intensity Variation Modeling Enhances Generalization in Neural Image Segmentation

Computational neuroscientists seeking to map the intricate wiring of the mammalian brain have long struggled with the severe performance degradation that occurs when deep learning segmentation models are transferred across differing imaging systems. In a study published in Brain Informatics, researchers led by Lin Cai, Ying Zhang, Quanwei Ding, Xiaojun Wang, Pei Sun, Shaoqun Zeng, and Tingwei Quan introduced SegAnyNeuron, an image segmentation network designed to maintain robust generalization across heterogeneous optical microscopy platforms by explicitly modeling voxel-scale image intensity variations. By extracting spatial orientation features from neurites and combining them with multi-scale dynamic signal properties, the method mitigates the substantial domain shifts caused by intensity discrepancies between foreground structures and image backgrounds. In experimental evaluations, the framework improved F1-scores by 7.9 percent to 22.8 percent compared to the standard UNet-3D baseline and outperformed existing state-of-the-art segmentation architectures by 6.0 percent to 9.1 percent across cross-modal neuronal datasets, presenting a biologically informed strategy to automate large-scale neuronal reconstructions.

Accurately determining fine neuronal morphology through three-dimensional reconstruction forms the foundation for characterizing structural connectivity within neural circuits, supported by imaging advances that capture whole mouse brains at sub-micron resolution. However, automated high-throughput data acquisition yields complex datasets marked by intricate branch structures, diverse molecular labeling, and labor-intensive tissue preparation protocols. While deep neural networks identify voxels occupied by neurites and enhance local signals to improve subsequent reconstruction, existing architectures suffer from a critical generalization bottleneck. Models optimized on specific datasets experience substantial performance drops when exposed to heterogeneous samples or alternate imaging systems, requiring manual re-annotation or complex weakly supervised retraining workflows. Because manual labeling is resource-intensive and weakly supervised iterative pipelines depend on reliable initial baselines, overcoming domain shifts caused by intensity discrepancies between foreground neurites and background voxels is essential to eliminate manual intervention bottlenecks in large-scale morphological connectomics.

To address these domain discrepancies without altering underlying deep backbones, the researchers analyzed structural signal behaviors distinguishing neurite foregrounds from surrounding backgrounds. Their quantitative analysis demonstrated that background regions exhibit highly uniform intensity distributions within local neighborhoods, with voxel values confined within narrow ranges due to the low-pass filtering properties of optical point spread functions. Conversely, genuine neurite regions display abrupt signal transitions that occupy broader dynamic ranges. Because neurites possess narrow physical radii, signal intensity decays exponentially as distance from the axonal centerline increases, dropping to ambient background noise levels once deviation exceeds three micrometers, corresponding to approximately one to two voxel spacings. This geometric constraint concentrates high-gradient features within a narrow zone near the axon center, whereas background regions exhibit neighborhood averaging marked by low gradients and high spatial homogeneity. Modeling these distinct physical characteristics provides a robust data-centric foundation to prevent imaging noise from disrupting segmentation accuracy.

Translating these physical observations into mathematical features begins with radial direction estimation, an operation that determines the principal orientation of filamentous structures for every voxel across three-dimensional space. The framework establishes a spherical coordinate system to uniformly sample 108 unit direction vectors, using twelve longitudinal divisions and nine latitudinal divisions. Each directional vector is transformed into a 13 by 13 by 13 voxel directional convolution kernel whose weights decay to zero for orientations deviating from the target vector. To enforce structural smoothness, each kernel is convolved with a 3 by 3 by 3 Gaussian kernel and normalized so its voxel sum equals one. Performing convolutions across the image identifies the vector yielding maximum accumulated intensity, defining the principal radial orientation at that point. Because neurites form cylindrical geometries whose cross-sectional areas are minimized along transverse planes, identifying this principal vector allows the system to determine the precise transverse plane where signal transitions from the neurite core to background levels are sharpest.

After establishing the principal radial orientation, SegAnyNeuron isolates two mutually orthogonal sampling directions within the transverse plane to capture localized intensity dynamics. Symmetrical intensity sampling extends five voxels in opposite directions along both orthogonal axes, generating two eleven-point profiles centered on the target voxel. The framework evaluates these profiles across seven progressively expanding intensity intervals defined by multiplying the global image standard deviation by a scaling parameter, configured as 0.4 during testing and varied between 0.1, 0.2, and 0.4 for training data augmentation. For each interval, the algorithm counts points whose intensities remain within threshold bounds along both axes, calculating a dynamic feature value scaled between zero and 255. In background areas, feature values exhibit abrupt shifts as thresholds expand, whereas signal regions demonstrate smooth, continuous response patterns. Generating seven distinct feature maps per image volume explicitly encodes multi-scale dynamic signal characteristics into auxiliary input channels that are directly concatenated with the original volumetric image.

The resulting multi-channel representation, combining raw image data with the seven intensity variation feature maps, was supplied to a standard UNet-3D architecture to ensure experimental outcomes reflected feature contributions rather than custom architectural modifications. Training utilized image blocks measuring 192 by 192 by 192 voxels, optimized via Adam with an initial learning rate of 0.0002, a batch size of one, and group normalization partitioned into eight groups. Model training ran for 200 epochs, taking approximately 13 hours on a workstation equipped with an Intel Core i7-6850K processor, 32 gigabytes of random-access memory, and an 11-gigabyte Nvidia GeForce GTX 1080Ti graphics processing unit running PyTorch. Because foreground neurites constitute a tiny fraction of total image volume, training utilized a composite loss balancing Dice loss and Focal loss. The Focal loss employed a squared exponent with a positive-negative sample balancing factor of 0.8, weighted by a coefficient of 50 to maintain effective gradient updates despite severe background class dominance.

Experimental evaluation was conducted across three distinct fluorescence micro-optical sectioning tomography datasets obtained from viral-tracer-labeled mouse brains across three laboratories, with voxel dimensions of 0.35 by 0.35 by 1.0 micrometers. These imaging series displayed substantial heterogeneity: dataset one exhibited high signal intensities averaging 2332 to 5123 with variances of 141 to 2882; dataset two showed stable intensities between 71 and 157 with variances of 11 to 210; and dataset three featured moderate fluctuations between 35 and 280 with variances of 12 to 473. In single-source trials, a model trained solely on dataset one achieved F1-scores of 90.6 percent on dataset two and 85.3 percent on dataset three, whereas the baseline UNet-3D yielded only 76.8 percent and 77.4 percent. When trained across a merged cohort of 108 image blocks and evaluated on 131 held-out test volumes, SegAnyNeuron achieved consistent F1-score improvements of 8.7 percent, 7.8 percent, and 3.9 percent across the respective test partitions, suppressing the false-positive detections that degraded baseline precision.

To evaluate cross-modality adaptability under severe domain shift, networks trained on fluorescence micro-optical sectioning tomography were applied directly to image volumes gathered from alternative microscopy modalities. The evaluation suite comprised 54 cropped volumes of 192 by 192 by 192 voxels from two-photon laser scanning microscopy and 30 volumes from light-sheet microscopy. On two-photon data, where baseline UNet-3D misclassified extensive background noise as connected neurite structures to record an F1-score of 62.2 percent, SegAnyNeuron attained an F1-score of 89.6 percent, representing a 27.4 percentage point gain. On light-sheet microscopy, where the baseline generated numerous false-positive detections and scored an F1-score of 76.4 percent, SegAnyNeuron reached 86.6 percent, achieving a 10.2 percentage point enhancement. These cross-system evaluations confirmed that modeling localized intensity variations equips the segmentation pipeline with domain-invariant representations that reliably preserve structural fidelity across disparate optical imaging technologies.

SegAnyNeuron was further benchmarked against contemporary volumetric segmentation networks, including nnFormer and UNETR++, trained on identical multi-source datasets. Although transformer-based models achieved high precision, they suffered from low recall due to underestimating fine neurite processes, whereas SegAnyNeuron established balanced precision and recall to deliver superior overall F1-scores across all cross-modal datasets. Systematic ablation experiments verified that removing radial direction estimation caused marked performance declines, while restricting dynamic signal features to a single scale triggered severe degradation under domain shift, with performance recovering as feature scales expanded from three to five intervals. Furthermore, controlled perturbation trials scaling background noise between factors of 0.2 and 0.8 demonstrated that while baseline UNet-3D performance fell from 97.1 percent at minimum noise to 83.0 percent at maximum noise, SegAnyNeuron maintained an F1-score of 92.2 percent, exhibiting a 9.2 percentage point margin that confirmed its resistance to imaging noise.

Computational profiling demonstrated that generating voxel-level directional features for a 192 by 192 by 192 block required approximately two seconds on a single graphics processing unit, with data transfer adding roughly four seconds before training ingestion. Because feature computation scales linearly with voxel volume and direction counts, block-wise processing permits seamless handling of terabyte-scale whole-brain datasets without exceeding memory bounds. Nevertheless, the researchers identified clear operational boundaries within the existing framework. The fixed radial modeling strategy struggles to capture structures with diameters substantially larger than typical neurite processes, such as thick blood vessels, and exhibits reduced reliability when segmenting neurites characterized by extremely low contrast and weak signal intensities. Future development will focus on implementing feature generation natively within Python to enable on-the-fly execution during training, incorporating adaptive sampling strategies that adjust to local structural dimensions, and exploring integration with emerging biomedical foundation models to broaden automated segmentation across diverse tubular biological architectures.

Subject of Research: Three-dimensional neuronal image segmentation and cross-domain generalization

Article Title: SegAnyNeuron: a neural image segmentation network with strong generalization performance by modeling image intensity variation

Article References: Cai, L., Zhang, Y., Ding, Q., Wang, X., Sun, P., Zeng, S., & Quan, T. (2026). SegAnyNeuron: a neural image segmentation network with strong generalization performance by modeling image intensity variation. Brain Informatics, 13(1), Article 11. https://doi.org/10.1186/s40708-026-00298-x

Image Credits: AI Generated

DOI: 10.1186/s40708-026-00298-x

Keywords: neuronal reconstruction, SegAnyNeuron, image segmentation, domain generalization, intensity variation, fMOST microscopy, two-photon microscopy, light-sheet microscopy, UNet-3D, radial direction estimation, dynamic signal features, voxel-scale modeling, multi-channel fusion, noise robustness, Dice loss, Focal loss, morphological analysis