Brain Volumetry Pipelines Are Not Interchangeable, Landmark Comparison Reveals

When a clinical trial reports that a drug slowed brain shrinkage, the number behind that claim usually comes from an automated segmentation pipeline: software that carves an MRI scan into gray matter, white matter, and deep structures like the hippocampus, then adds up the voxels. A new head-to-head comparison of six widely used brain volumetry pipelines, published in Frontiers in Neuroscience, delivers a sobering message for anyone interpreting such numbers: the same brain, measured by different established tools, can yield systematically different volumes, and the differences are large enough to matter for disease monitoring and drug development.

The study, led by Nedim Šišić, Muhamed Barakovic, Stefano Magon, and Susanna Gobbi with collaborators at the University of Sherbrooke and Roche, compared FreeSurfer versions 6.0 and 7.2, SAMSEG, FastSurfer v1, SIENAX/SIENA-XL, and CAT12.8.2 across three cohorts: 19 or 20 healthy adults scanned five times at four-week intervals, 20 people with relapsing-remitting multiple sclerosis scanned on the same schedule, and 50 participants drawn from the placebo arm of the GRADUATE I phase III Alzheimer’s disease trial, imaged at baseline and weeks 48, 104, and 116. The researchers normalized every tissue volume to intracranial volume, expressed as a percentage of ICV, so that head-size differences could not masquerade as pipeline effects.

The headline finding is stark: baseline volume estimates differed significantly across pipelines in 11 of 12 cohort-by-tissue comparisons, with false discovery rate-adjusted p-values below 0.001. The only exception was white matter in the multiple sclerosis cohort, where pairwise differences ranged from −0.27 to 0.23 percent ICV and none survived correction. Gray matter showed the largest offsets. CAT12 consistently produced the highest normalized gray matter estimates and SIENAX the lowest, with differences of 6.59 percent ICV in healthy controls, 6.27 in multiple sclerosis, and 7.60 in Alzheimer’s disease. For context, the expected annual whole-brain atrophy in Alzheimer’s disease is on the order of one to two percent of brain volume; a seven-unit gray matter offset between pipelines dwarfs the biological signal trials are designed to detect.

The subcortical structures told a similarly method-dependent story, but with the rankings reversed. SAMSEG produced the highest hippocampal estimates while CAT12 produced the lowest, with the largest pairwise offsets reaching 0.11 to 0.12 percent ICV. For the thalamus, the pattern flipped: CAT12 produced the highest estimates and SAMSEG the lowest, with differences of 0.13 to 0.18 percent ICV. These reversals are not bugs but consequences of design. FreeSurfer and FastSurfer emphasize anatomically labeled cortical and subcortical morphometry using surface reconstruction; SAMSEG applies a sequence-adaptive Bayesian generative model to voxel-based segmentation; SIENAX was built primarily for normalized global tissue volumes using brain extraction, registration-based head-size normalization, and partial-volume classification; and CAT12 relies on tissue-probability modeling and spatial normalization, with regional volumes derived from the Neuromorphometrics atlas rather than from its global gray matter estimate. Different tissue priors, partial-volume modeling, brain extraction, registration, and cortical-boundary placement all push the numbers in different directions.

Agreement between pipelines, quantified with intraclass correlation coefficients, depended strongly on tissue and cohort. White matter showed the strongest concordance overall, especially in healthy controls and multiple sclerosis, though even white matter diverged in Alzheimer’s disease, where SAMSEG produced higher estimates than SIENAX by 2.53 percent ICV and than CAT12 by 2.18. Gray matter agreement was more variable, particularly when SIENAX or CAT12 was compared with the FreeSurfer-like pipelines. FastSurfer agreed strongly with FreeSurfer, which was expected given that its deep-learning segmentation model was trained to reproduce FreeSurfer-style labels before a surface-reconstruction module refines the volumes. Subcortical structures showed the greatest method dependence, with hippocampal agreement strongest among FreeSurfer and FastSurfer outputs and SAMSEG and CAT12 differing more substantially from the rest.

The researchers then asked whether expressing follow-up scans as change from baseline, rather than absolute volume, could reconcile the pipelines. Mixed-effects models of percentage change from baseline found no significant pipeline-by-timepoint interaction after FDR correction in any tissue, cohort, or processing mode, with interaction F-values ranging from 0.12 to 1.61 and all adjusted p-values equal to 1.00. In other words, once each subject’s own baseline anchored the comparison, the pipelines traced broadly parallel longitudinal trajectories. This is genuinely reassuring: it suggests that the systematic offsets are largely additive and that relative change estimates are more comparable than absolute volumes. However, significant pipeline main effects persisted in selected tissues, especially in the Alzheimer’s cohort, where seven of eight models showed pipeline differences in the average magnitude of estimated change. Most pipelines estimated progressive gray matter decline, but SIENA-XL estimated a numerically smaller change, and for white matter SAMSEG estimated positive or less negative change where most other pipelines estimated decline. Cross-sectional agreement, the authors conclude, does not guarantee concordant longitudinal trajectories.

The study also tested the theoretical advantage of dedicated longitudinal processing, which builds an unbiased within-subject template from all available timepoints and reprocesses each scan against it, rather than segmenting each timepoint independently. The benefit was real but strikingly uneven. In healthy controls, longitudinal processing reduced the within-subject standard deviation of thalamic change estimates by 58.1 percent with FreeSurfer v6 and 63.6 percent with FreeSurfer v7.2, and hippocampal variability fell by 34.1 and 55.7 percent respectively. FreeSurfer v7.2 hippocampal intraclass correlations rose from 0.932 with cross-sectional processing to 0.991 with longitudinal processing. But the effect was not uniform: SAMSEG longitudinal increased gray and white matter variability by roughly 10 percent, SIENAX longitudinally increased white matter variability by 13.8 percent, and CAT12 thalamic variability rose by 49.5 percent in the multiple sclerosis cohort. Across all settings, the effect of longitudinal processing ranged from a 49.5 percent increase to a 63.6 percent reduction in within-subject variability.

An exploratory calibration analysis examined whether the systematic offsets could be mathematically bridged. Using Deming regression fitted to healthy-control baseline volumes, the researchers transformed one pipeline’s measurements onto another’s scale and applied the coefficients unchanged to the multiple sclerosis and Alzheimer’s cohorts. Across 164 transfers, median absolute bias fell from 0.22 to 0.11 percent ICV and median concordance rose from 0.71 to 0.82. But performance was strongly cohort dependent: calibration improved mean absolute error in 82.9 percent of comparisons when transferred to the closely matched multiple sclerosis cohort, which shared age, sex, scanner, and protocol with the calibration sample, yet in the Alzheimer’s cohort median MAE actually increased from 0.73 to 0.86 percent ICV because some mappings overcorrected. A universal conversion table, the authors suggest, is unlikely to harmonize heterogeneous multicenter datasets reliably.

The multiple sclerosis cohort required lesion filling before segmentation, a step in which white matter lesions identified on FLAIR images are replaced with intensities resembling neighboring normal-appearing white matter. This is standard practice because lesions distort tissue classification, but it introduces its own pipeline dependency: differences between pipelines in multiple sclerosis may reflect not only segmentation algorithms but also how each method responds to lesion-filled images, particularly near the gray-white interface where lesions can shift intensity normalization and cortical-boundary placement.

The practical implications are direct. Pipeline outputs should not be treated as interchangeable on their native scales, particularly for gray matter, hippocampus, and thalamus. Even within a single pipeline family, FreeSurfer v6 and v7.2 agreed relatively well but were not identical, so switching software versions mid-study could introduce a measurement shift indistinguishable from biological change. The authors recommend that all timepoints from a participant be processed with the same pipeline, version, processing stream, command options, and computational environment, and that longitudinal trials predefine the pipeline, version, processing mode, ICV normalization strategy, lesion-processing approach, and quality-control criteria before data collection begins. Where existing datasets contain mixed versions, the preferred remedy is complete reprocessing with one prespecified version, with any bridging analysis quantifying tissue- and cohort-specific bias first. The study had no ground truth against which to judge absolute accuracy, and the healthy and multiple sclerosis cohorts were small and single-site, so the findings speak to relative agreement rather than anatomical correctness. But the core conclusion stands: in brain volumetry, the measuring instrument is part of the measurement, and consistency, not interchangeability, remains the most robust foundation for longitudinal studies and clinical trials.

Subject of Research: Brain volumetry pipeline agreement and longitudinal repeatability

Article Title: Agreement and longitudinal repeatability of established brain volumetry pipelines across healthy controls, multiple sclerosis, and Alzheimer's disease

Article References: Agreement and longitudinal repeatability of established brain volumetry pipelines across healthy controls, multiple sclerosis, and Alzheimer's disease. (n.d.). https://doi.org/10.3389/fnins.2026.1917471

Image Credits: AI Generated

DOI: 10.3389/fnins.2026.1917471

Keywords: brain segmentation, brain volumetry, MRI, FreeSurfer, SAMSEG, FastSurfer, SIENAX, CAT12, multiple sclerosis, Alzheimer's disease, longitudinal processing, repeatability