The case study at the heart of the work is the proposed FASER calorimeter, or FASERCAL, an upgrade concept for the FASER experiment at CERN, which sits in the path of the collider’s forward neutrino beam. Its primary interaction volume, a highly granular three-dimensional calorimeter known as 3DCAL, comprises more than 460,000 scintillator voxels arranged in ten longitudinal modules, only a small fraction of which light up in any given event. Downstream, an electromagnetic calorimeter, a hadronic calorimeter and a muon spectrometer complete the detector. The beam contains electron, muon and tau neutrinos, and the analyses target both charged-current and neutral-current interactions. The resulting event images contain dense shower cores, extended secondary tracks, partial containment and deeply ambiguous local configurations. In this regime, the authors argue, machine learning is not an optional enhancement to an otherwise tractable reconstruction problem; it is a mandatory prerequisite for extracting any event-level physics at all.

The team’s approach borrows a page from the masked-autoencoder techniques that have transformed language and vision modelling. During pretraining, 75 percent of the occupied calorimeter patches are hidden, and a lightweight decoder must reconstruct the missing voxel-level occupancy and charge from whatever remains. To succeed, the encoder is forced to infer non-local spatial correlations, recovering broad shower envelopes, elongated track-like structures and coherent energy flow into the masked regions. On top of this reconstruction objective, the researchers added what they call relational voxel-level tasks: predicting, for each visible patch, whether it contains ghost activity with no matched true particle, whether it belongs to background, primary or secondary interaction hierarchy, and whether its deposits are electromagnetic, muonic or hadronic in character. These targets are especially demanding because a single reconstructed voxel can receive contributions from multiple true particles, yielding soft class mixtures rather than clean one-hot assignments.

The architecture itself is tailored to the detector’s heterogeneity. Sparse three-dimensional convolutions convert the 3DCAL and hadronic-calorimeter voxel grids into patch tokens, so that computational cost scales with detector occupancy rather than with the total instrumented volume. Module-aware self-attention blocks then capture local shower patterns within each of the ten detector modules before a Perceiver-IO bottleneck fuses the calorimetric tokens with compact representations of the electromagnetic calorimeter’s energy matrix and the variable-length muon-spectrometer tracks. The full fine-tuned model contains roughly 32 million trainable parameters. Pretraining ran on eight NVIDIA GH200 GPUs, with the masked-reconstruction phase consuming 584 GPU-hours and the relational continuation adding a further 427, an incremental cost of 73 percent for targeted gains in the hardest channels.

The downstream results are striking, and the pattern of improvement is as informative as its magnitude. For the dominant electron-neutrino charged-current channel, the area under the receiver operating characteristic curve rose from 0.968 for a model trained from scratch to 0.985 with full pretraining; for muon neutrinos, from 0.909 to 0.958; and for neutral-current events, from 0.885 to 0.947. But the most consequential gains appeared in the tau-neutrino channels, precisely where dense overlap, secondary activity and partial containment make classification hardest. For hadronic tau decays, the maximum figure of merit, a standard signal-to-background metric, jumped from 1.58 to 4.58. For the electron tau channel it nearly quadrupled, from 0.38 to 1.35. Charmed-quark identification showed the same structure, with the relational objectives disproportionately improving the lower-yield channels that matter most for the detector’s physics reach.

Regression tasks benefited as well. Primary-vertex reconstruction showed the clearest and most uniform effect, with error distributions for the pretrained models shifted towards smaller values and their interquartile ranges tightened across essentially all event categories. Visible energy, missing transverse momentum and jet momentum also improved, though less uniformly, and at the smallest label budgets of a few hundred events pretraining was not consistently advantageous for every continuous target. The authors are careful about such nuances, presenting the relational objective as generally superior across the task suite rather than universally dominant, and noting that masked reconstruction alone accounts for most of the gain in several regression quantities.

Perhaps the most operationally significant finding concerns data efficiency. Labelled events in an energy-frontier neutrino programme are not a cheap by-product: rare channels require dedicated simulation campaigns, systematic variations are costly, and many target quantities depend on expensive truth association. The data-efficiency study varied labelled training budgets from one hundred to one hundred thousand events while keeping validation and test sets fixed. With roughly one thousand labelled events, the fully pretrained encoder reached a flavour-classification macro-AUROC of 0.818, exceeding the 0.807 achieved by a scratch-trained model with ten times more data. Vertex reconstruction showed a similar compression, with the mean error falling from 239 millimetres for scratch training to 100 millimetres with full pretraining at the same budget. That leftward shift of the performance frontier changes not only how efficiently models can be trained but which physics studies are practical to pursue.

The team also probed whether the learned representation carries meaning beyond its original detector. Saliency maps concentrated attribution near the interaction region and along the main downstream shower and track structures rather than spreading it diffusely. Latent-space projections revealed cleaner flavour groupings and smoother visible-energy progressions for the pretrained variants. Detector-subsystem ablations recovered a physically plausible division of labour: removing the 3DCAL collapsed flavour performance and inflated the vertex error from 21 millimetres to nearly a metre, while dropping the hadronic calorimeter most strongly affected hadronic discrimination, the muon spectrometer degraded muonic channels, and removing the electromagnetic calorimeter worsened energy measurements. A coherent energy-scale stress test, shifting all calorimeter responses by up to ten percent at inference time, produced only mild drifts, suggesting the representation is not acutely brittle to global calibration biases.

Transfer beyond the source domain provided further evidence of reusability. On a public fine-grained plastic-scintillator benchmark at giga-electronvolt energies, the pretrained encoder improved classification diagonals for protons, charged pions, muons and electrons over scratch training, surpassing the strongest published baseline for three of the four species. On PILArNet, a public liquid-argon time-projection-chamber dataset representing a far greater technological and energy shift, the tera-electronvolt-scale encoder adapted to particle-level classification and, on the multi-particle task, edged past the best published ensemble results in both accuracy and area under the curve. A matched alternative-generator stress test, evaluating the same checkpoints on NuWro events matched event-by-event to the GENIE training sample, showed stable flavour and kinematic performance while exposing charm tagging as generator sensitive, a boundary the authors interpret as evidence of task-dependent robustness rather than generator independence.

The researchers are explicit about limits: the study is simulation based, the relational targets inherit simulation truth, and a full detector foundation model would require broader coverage, stronger systematic validation and ultimately calibration on experimental data. They reserve the phrase foundation-style for what they have demonstrated: a single reusable encoder, trained with objectives independent of downstream labels, adapted to many physics tasks, effective with few labels and transferable across detector technologies. For energy-frontier neutrino physics, where event complexity makes artificial intelligence a requirement rather than an optional convenience, that is already a substantive result, and it outlines a concrete path towards the general-purpose detector encoders that future experiments may one day take for granted.

Subject of Research: Self-supervised pretraining of sparse vision transformer models for energy-frontier heterogeneous neutrino detector event reconstruction

Article Title: Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pretraining

Article References: Alonso-Monsalve, S., Cufino, F., Kose, U., Mascellani, A., & Rubbia, A. (2026). Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pretraining. Nature Machine Intelligence. https://doi.org/10.1038/s42256-026-01309-6

Image Credits: AI Generated

DOI: 10.1038/s42256-026-01309-6

Keywords: neutrino physics, self-supervised learning, masked autoencoders, machine learning, particle detectors, FASER, CERN, vision transformers, transfer learning, event reconstruction, Nature Machine Intelligence, foundation models