Transformers Are Rewriting How We Read the Human Brain

The same architecture that powers ChatGPT is quietly transforming one of neuroscience’s hardest problems: making sense of the torrent of data produced when a human brain is scanned. A new review published in Frontiers in Neuroinformatics maps, for the first time in a unified framework, how Transformer models have swept through functional magnetic resonance imaging (fMRI) analysis, and argues that they may be the key to building general-purpose “brain foundation models” that could one day help diagnose Alzheimer’s, depression, autism, and more.

fMRI is a deceptively tricky technology. It does not measure neural activity directly; instead, it tracks the blood oxygen level-dependent (BOLD) signal, an indirect proxy for where neurons are firing. A single scan yields a four-dimensional dataset, three spatial dimensions stacked through time, containing tens of thousands to hundreds of thousands of voxels, each with its own slowly fluctuating time series. The signal is noisy, contaminated by head motion, breathing, and scanner drift, and it varies enormously from person to person and from scanner to scanner. Traditional statistical methods, such as generalized linear models and static functional connectivity measures, assume linearity or average activity over an entire scan, discarding precisely the dynamic brain reorganization that scientists most want to understand.

Enter the Transformer. Introduced by Vaswani and colleagues in 2017 for natural language processing, the architecture rests on a deceptively simple mechanism: self-attention, which computes relationships between every pair of elements in a sequence, allowing the model to capture long-range dependencies without the fixed local windows of convolutional networks or the sequential bottleneck of recurrent ones. When researchers realized that a brain scan could be chopped into tokens just like a sentence, in time, in space, or both, the field began to reorganize itself around the idea. The review’s authors, led by Xiaomin Yang and Bocheng Wang of the Communication University of Zhejiang, sort this rapidly growing literature into four paradigms: temporal modeling, spatial modeling, joint spatiotemporal modeling, and self-supervised learning.

Temporal models treat fMRI as a multivariate time series. Brain atlases such as AAL or Schaefer collapse the brain into a set of regions of interest, and each moment in the scan becomes a token representing the brain’s global state. Self-attention then builds a matrix of dependencies across all time points, capturing the slow-frequency fluctuations below 0.1 Hertz that carry biologically meaningful dynamics, from transient synchronization between networks to spontaneous microstate shifts at rest. Models like BolT use windowed attention to make long sequences computationally tractable, and have delivered strong results classifying major depressive disorder, while transfer-learning approaches have adapted Vision Transformer weights pretrained on images to fMRI time series, improving detection of mild cognitive impairment and Alzheimer’s disease.

Spatial models, by contrast, ask how brain regions collaborate regardless of when. Here the field has discovered something profound: functional brain networks are fundamentally non-Euclidean. Regions that work together tightly, such as nodes of the default mode network spanning the front and back of the brain, are often physically far apart, which is exactly where convolutional networks with local receptive fields fail. A spatial Transformer’s attention matrix effectively learns a dynamic functional connectivity map from the data itself, without the predefined graph structures that constrain graph neural networks. Landmark systems such as BrainNetTF treat each anatomical region as a token and model the whole-brain topology end to end, while architectures like Brainformer preserve 3D coordinates and VoxelFormer merges semantically similar tokens to tame the computational explosion of voxel-level data.

But the brain is neither purely temporal nor purely spatial. Network neuroscience insists that brain function is a dynamic system in which space and time are deeply intertwined, and forcing the data into a single dimension inevitably loses something. Joint spatiotemporal modeling, the review’s third paradigm, tackles this directly. Factorized approaches such as Twin-Transformer run parallel spatial and temporal branches, while unified 4D models like SwiFT extend the shifted-window mechanism of the Swin Transformer into four dimensions, computing attention locally in space-time while preserving global information flow. Newer systems such as NeuroSTORM abandon predefined atlases entirely, extracting fine-grained, atlas-free features across the whole 4D volume. The payoff is the ability to explicitly model dynamic functional connectivity: to see not just whether a brain is abnormal, but when, where, and how its networks reorganize, a shift the authors describe as moving from black-box classifiers to dynamic process analyzers.

The fourth and arguably most consequential paradigm is self-supervised learning, which attacks fMRI’s most stubborn problem: high-dimensional data, tiny samples. Scanning is expensive and clinical cohorts are often only a few hundred subjects, yet voxel-level feature counts run into the hundreds of thousands, a recipe for overfitting. Self-supervised methods sidestep the need for labels by inventing pretext tasks. Masked autoencoders hide large chunks of the input, up to 75 percent of spatiotemporal patches in some 3D implementations, and force the model to reconstruct the missing signal, compelling it to internalize the constraints of brain anatomy and network organization. Contrastive learning takes a different route, teaching models to recognize that two augmented views of the same subject’s scan are more alike than scans of different people, sharpening sensitivity to individual differences.

These techniques are converging on something bigger: brain foundation models. BrainLM, pretrained on roughly 80,000 scans from the UK Biobank, supports zero-shot inference of brain latent states and few-shot adaptation to clinical prediction tasks such as depression and anxiety. BrainSymphony incorporates structural connectivity priors during pretraining, aligning learned representations with the brain’s true structure-function coupling. Recent work has even demonstrated that scaling laws, the predictable performance gains that transformed language models, hold for brain imaging models, a finding the review highlights as evidence that the field is entering its own era of large-scale pretraining.

The clinical stakes are high. Spatiotemporal graph Transformers have outperformed conventional deep learning in distinguishing Alzheimer’s patients from healthy controls; mixture-of-experts architectures fuse fMRI, structural MRI, and diffusion imaging to separate Parkinson’s disease from its prodromal stage; and multi-atlas dynamic connectivity Transformers are pushing autism classification forward. On the decoding side, Transformer-based models have reconstructed photorealistic images from brain activity and can even generate natural language descriptions of what a person is viewing. The same global modeling capability that makes Transformers good at diagnosis also makes them more robust to the scanner-to-scanner variability that has long plagued multi-site clinical studies, though domain shift remains far from solved.

The review is careful not to oversell. Self-attention scales quadratically with sequence length, making whole-brain 4D modeling ferociously expensive, and window approximations trade away some long-range fidelity. More troubling is interpretability: a lively technical debate over whether attention weights explain anything at all cautions against reading attention maps as maps of neural causality. The statistical associations a model learns are not anatomical connections, and whether pretrained representations truly reflect functional brain organization still demands validation with neuroscience methods. The authors argue the path forward lies in hybrid strategies, embedding structural connectivity, anatomical priors, and causal inference techniques into attention mechanisms, and in building unified data standards so that the next generation of foundation models can generalize across centers. If they succeed, the technology behind chatbots may become the lens through which medicine finally reads the brain in motion.

Subject of Research: Transformer-based machine learning methods for fMRI brain data analysis

Article Title: Transformers for fMRI analysis: from spatiotemporal modeling to brain foundation models

Article References: Yang, X., Li, Y., Zhang, X., & Wang, B. (2026). Transformers for fMRI analysis: from spatiotemporal modeling to brain foundation models. Frontiers in Neuroinformatics, 20, Article 1869687. https://doi.org/10.3389/fninf.2026.1869687

Image Credits: AI Generated

DOI: 10.3389/fninf.2026.1869687

Keywords: fMRI, Transformer, self-attention, brain foundation models, functional connectivity, self-supervised learning, masked autoencoders, neuroimaging, brain decoding, BOLD signal, deep learning, clinical diagnosis