In a study published in the International Journal of Computer Assisted Radiology and Surgery, Jingjing Peng of King’s College London and colleagues, working with neurosurgeons at the National Hospital for Neurology and Neurosurgery and King’s College Hospital, introduced NeuralShift, a U-Net-based deep learning framework that predicts a dense brain displacement field using only preoperative MRI and the side of the planned temporal lobe resection. The work targets epilepsy surgery, in which patients with drug-resistant seizures undergo removal of a temporal lobe to eliminate the seizure focus. Because these procedures follow relatively standardised corridors and positioning, they provided an ideal proving ground for a data-driven model of intraoperative deformation.

The clinical stakes are considerable. Neuronavigation aligns preoperative images with the patient’s anatomy in the operating theatre, guiding the surgeon toward the resection target while avoiding critical structures such as the optic radiation, the fibre bundle that carries visual information and sits dangerously close to the surgical corridor. When the brain shifts, the map becomes wrong, and the surgeon may be navigating a phantom. Intraoperative MRI can capture the deformation directly, but it is expensive, time-consuming, disruptive to the surgical workflow, and available in only a minority of operating rooms worldwide. Intraoperative ultrasound offers a faster and cheaper alternative but produces lower-contrast, operator-dependent images that provide limited objective information about deeper structures.

NeuralShift takes a deliberately different route. Rather than attempting to image the brain during surgery, the model learns a cohort-level prior: the statistical pattern of how temporal lobes deform after resection in this patient population. The architecture is built on U-Net, a convolutional network originally designed for biomedical image segmentation, and it produces three outputs simultaneously: a dense three-dimensional displacement field that warps the preoperative scan toward the intraoperative configuration, a prediction of the deformed brain mask, and the corresponding signed distance function, a continuous encoding of the distance to the brain boundary that supplies strong geometric constraints on global shape.

One of the study’s most technically interesting choices lies in how the network was trained. Voxel-wise physical measurements of brain deformation are essentially impossible to obtain, so the researchers used a registration-based surrogate supervision strategy. Each preoperative MRI was registered to its paired post-resection intraoperative MRI using the Fast Free-Form Deformation algorithm implemented in NiftyReg, and the resulting deformation field served as the regression target. The loss function was correspondingly sophisticated, combining a Cartesian mean squared error on displacement vectors with an auxiliary spherical-coordinate loss that penalises errors in the direction and magnitude of each displacement independently, alongside Dice and edge losses for the mask and a mean squared error for the signed distance function. This multi-task objective forces the network to respect both local deformation physics and global brain geometry at the same time.

Careful preprocessing was required to make the paired scans comparable. Intraoperative images contain a surgical cavity and intensity heterogeneity that break standard registration tools, so the team designed a bespoke pipeline. Anatomists manually identified three landmarks on each scan, the anterior commissure, posterior commissure, and intercomissural height, defining a subject-specific coordinate system that standardised orientation before both images were rigidly and affinely registered into the Montreal Neurological Institute template space. Critically, the model was given only a binary hemisphere indicator encoding resection laterality, deliberately excluding any information about the resection cavity itself, because using the cavity segmented from the intraoperative scan would constitute target leakage, revealing an outcome that cannot be known before surgery.

The evaluation was conducted with notable methodological rigour for a feasibility study. From 98 paired preoperative and intraoperative MRI cases, eight were reserved as a fixed validation set for checkpoint selection, and the remaining 90 were divided into nine disjoint test folds of ten cases each. Nine independently initialised models were trained, each evaluated on a fold it had never seen, yielding one out-of-fold prediction for every subject. The results showed consistent improvement: the predicted brain masks achieved an unweighted mean Dice score of 0.97 across folds, up from 0.93 for the undeformed preoperative anatomy, while registration-referenced target registration error at clinically chosen landmarks, including the frontal operculum, the insula, the aqueduct of Sylvius, the corpus callosum, and the optic tract, fell from a range of 1.46 to 4.76 millimetres down to 1.12 to 3.05 millimetres after applying the predicted field.

The authors are unusually candid about the limits of these numbers. Because the landmark reference positions were generated by the same registration algorithm used as the training target, the error reduction measures how faithfully the network reproduces the registration surrogate, not independently verified physical accuracy. The Dice scores describe overlap with masks produced within the study’s own preprocessing workflow rather than manually delineated reference standards. No patient-specific biomechanical model was run on the same cohort, so no claims of equivalence or superiority to physics-based approaches are made. The study is also confined to a single centre with a homogeneous surgical population, meaning the learned deformation prior may not transfer to tumour resections, where oedema, craniotomy geometry, and lesion diversity introduce far greater variability.

Even so, the conceptual contribution is significant. Biomechanical models of brain shift, typically built on finite element methods, require assumptions about tissue elasticity, boundary conditions, loading, and resection geometry, along with a numerical solve that can burden the surgical workflow. NeuralShift, once trained, produces a deformation prior through a single feed-forward pass, without patient-specific meshing or parameter assignment. The trade-off is the absence of explicit physical guarantees and dependence on the deformation patterns represented in the training cohort. The authors position the two paradigms as complementary rather than competing, and they outline a compelling hybrid future in which the learned prior is updated in real time with sparse intraoperative observations such as ultrasound or tracked cortical surface points, preserving rapid inference while adapting to the actual surgical state.

The road ahead includes prospectively recorded resection plans that could localise expected tissue removal more precisely than a hemisphere indicator, explicit separation of true mechanical deformation from the volume change of the resection cavity itself, uncertainty quantification, and multi-centre validation with independently delineated intraoperative surfaces. The code will be released publicly after acceptance. If those steps succeed, the idea that a neural network can anticipate the brain’s silent drift before surgery begins could mark a turning point in image-guided neurosurgery, turning one of the operating room’s oldest frustrations into a solved prediction problem.

Subject of Research: Deep learning prediction of intraoperative brain shift in temporal lobe resection for epilepsy surgery

Article Title: From pre- to intra-operative MRI: predicting brain shift in temporal lobe resection for epilepsy surgery

Article References: Peng, J., Fiore, G., Liu, Y., Ellum, K., Dasgupta, D., Ashkan, K., McEvoy, A., Miserocchi, A., Ourselin, S., Duncan, J., & Granados, A. (2026). From pre- to intra-operative MRI: predicting brain shift in temporal lobe resection for epilepsy surgery. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03794-x

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03794-x

Keywords: brain shift, NeuralShift, U-Net, intraoperative MRI, preoperative MRI, epilepsy surgery, temporal lobe resection, neuronavigation, deep learning, displacement field, target registration error, computer-assisted surgery

The stakes could hardly be higher. In neurosurgery, millimetric margins separate eloquent cortex, major blood vessels and white matter tracts from tumour tissue, and prior studies have documented that inadvertent injury to eloquent cortical regions during tumour resection produces permanent neurological deficits in a substantial proportion of patients. Errors exceeding two to three millimetres significantly increase the risk of vascular injury, incomplete resection and neurological deterioration, while deep targets such as the basal ganglia, thalamic nuclei and brainstem demand sub-millimetre precision. In deep brain stimulation surgery, targeting deviations of just one to two millimetres can reduce clinical benefit and provoke stimulation-induced adverse effects. Against these requirements, the errors reported for AR/MR neurosurgical applications—translational errors ranging from 0.62 to 6.93 mm and angular errors from 1.32 to 6.80 degrees depending on hardware and registration method—are sobering.

The problem, the researchers argue, is that the conventional yardsticks of navigation accuracy are fundamentally static. Fiducial Registration Error (FRE), the root mean square distance between corresponding markers after registration, and Target Registration Error (TRE), which measures accuracy at anatomical positions not used in the registration, are both discrete-point measurements computed at a single moment. Neither can capture the dynamic degradation of registration quality during an operation, which can be driven by tracking system drift from thermal effects, electromagnetic interference and line-of-sight occlusions; by patient micro-movements despite rigid cranial fixation; and by brain shift—the displacement of brain tissue caused by cerebrospinal fluid drainage, gravity and tumour resection, which can exceed 10 millimetres. Intraoperative MRI and ultrasound can partially compensate, but these modalities are specialised, interrupt the surgical workflow and are not universally available.

RQI attacks this gap with a deliberately lightweight computer vision pipeline that runs entirely from the headset’s own camera feed. The experimental platform paired a Varjo XR-3 head-mounted display—offering 2880 by 2720 pixels per eye at a 90 Hz refresh rate—with six HTC Vive Lighthouse base stations providing sub-millimetre optical tracking. A ceramic skull phantom, dimensioned to match adult human skull morphometry, served as the anatomical reference, and its virtual counterpart was reconstructed from computed tomography segmentation at 0.5 mm voxel resolution, imported into Unreal Engine 5.0.3 and rendered as a semi-transparent cyan overlay. Spatial registration between the virtual and physical coordinate systems was anchored to a printed 2D ArUco fiducial marker, detected by the headset’s stereo cameras at 30 Hz to compute the six-degree-of-freedom transformation in real time.

The computation itself proceeds in five stages on each passthrough frame. After frame acquisition at 30 Hz and a resolution of 320 by 160 pixels, the skull region is segmented from the background by adaptive thresholding, and Canny edge detection extracts the contours of both the physical phantom and the virtual overlay. Because the virtual model is rendered in cyan—a hue absent from the bone-white phantom—each detected edge pixel can be classified by examining its neighbourhood in the original RGB frame, yielding separate binary maps of physical and virtual boundaries. For every pixel on the physical boundary, the minimum Euclidean distance to the nearest virtual boundary pixel is then computed using OpenCV’s distance transform. Finally, RQI is expressed as the percentage of physical boundary pixels whose displacement exceeds a proximity threshold: a perfectly registered system scores 0 percent, while progressive misregistration drives the value toward 100 percent. Higher RQI, in other words, means worse alignment.

Choosing that proximity threshold was a careful balancing act. A systematic sensitivity analysis across thresholds from 2 to 12 pixels showed that values below 4 pixels produced unstable readings inflated by sub-pixel edge localisation noise, while values above 8 pixels compressed the RQI distribution toward zero and made the metric insensitive to clinically relevant degradation. The selected threshold of 6 pixels—roughly 8 millimetres at the 40 cm working distance, where one pixel corresponds to approximately 1.3 mm—maximised the Pearson correlation with TRE and was independently supported by receiver operating characteristic analysis, discriminating acceptable from degraded registration with a Youden index of 0.87, sensitivity of 0.93 and specificity of 0.94. The authors candidly note that because the threshold was tuned on the same dataset used to characterise the metric, the values carry an optimistic bias and must be confirmed on independent data.

The validation results were striking. Across fifteen experimental trials and seventy-five viewing configurations, RQI correlated strongly with both established metrics: r = 0.89 with FRE (95% CI 0.69–0.96, p < 0.001) and r = 0.93 with TRE (95% CI 0.80–0.98, p < 0.001), the latter meaning that 87 percent of the variance in target accuracy was explained by the surface-wide pixel misalignment. Regression analysis quantified the relationship in practical terms: each 1 percent increase in RQI corresponded to a 0.195 mm increase in FRE and a 0.244 mm increase in TRE. Spearman rank correlations confirmed the relationship was monotonic across the full experimental range—crucial, because for intraoperative monitoring, detecting that registration quality has degraded matters more than knowing the absolute error. Mean FRE was 2.95 ± 0.48 mm and mean TRE 3.78 ± 0.57 mm, while mean RQI was 3.28 ± 1.37 percent.

Speed and usability were also put to the test, with encouraging results. The entire image analysis pipeline averaged about 10 milliseconds per frame on a CPU alongside the GPU-bound rendering, achieving real-time computation at 30 Hz with no dropped frames and no disruption to workflow. A linear mixed effects model across the 75 viewing configurations revealed that viewing angle significantly influenced RQI (p < 0.001, partial η² = 0.49): displacement was lowest at the frontal view and rose at oblique, lateral, posterior and contralateral perspectives, a geometrically predictable effect since errors perpendicular to the viewing axis project maximally. The authors suggest lateral views may serve as a more sensitive early warning of degradation, and that angle-dependent normalisation could improve alert specificity. Three board-certified neurosurgeons rated the system on the System Usability Scale at a mean of 79.0 ± 4.7, well above the acceptability threshold of 68.

The team is equally forthright about limitations. Validation was confined to a rigid phantom under controlled laboratory conditions and cannot account for brain shift, soft tissue deformation, pulsatile brain motion or variable illumination. A particularly important caveat is brain sag—the gravity-driven displacement of the brain after dural opening—which can migrate deep structures by several millimetres while the exposed cortical surface remains comparatively stable; a surface-based metric could then report acceptable alignment even as the true target drifts. The interpretability ranges offered in the study, with band boundaries at 2, 4 and 6 percent RQI, are explicitly exploratory research guidance rather than validated clinical decision boundaries, and the authors stress that RQI is not yet validated for intraoperative clinical decision-making. The most important next step, they write, is a dedicated experiment applying translations and rotations of predetermined magnitude to the registration transform to establish the metric’s sensitivity, linearity, repeatability and failure modes, followed by validation under clinical conditions and integration with intraoperative imaging.

Even so, the implications extend well beyond the cranial theatre. The authors point out that the absence of continuous, marker-free registration monitoring is equally pertinent to spinal navigation, where vertebral mobility between preoperative imaging and the operative position is a recognised source of drift, and where AR/MR guidance and robotic platforms are increasingly used for pedicle screw placement requiring sub-millimetre accuracy. A surface-based stability index computed on exposed bony landmarks could provide intraoperative feedback in those workflows too. If subsequent clinical studies confirm the promise of this proof of concept, the humble act of continuously comparing a cyan wireframe to the anatomy beneath it could become a quiet but essential guardian—watching, frame by frame, that the surgeon’s virtual map never silently stops matching the territory.

Subject of Research: A real-time metric for continuous monitoring of registration stability in mixed reality neurosurgical navigation

Article Title: A real-time metric for quantifying registration stability in mixed reality neurosurgical navigation

Article References: Fontana, C., De Notaris, M., Iaconetta, G., & Cappetti, N. (2026). A real-time metric for quantifying registration stability in mixed reality neurosurgical navigation. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03792-z

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03792-z

Keywords: mixed reality, neurosurgery, neuronavigation, registration accuracy, augmented reality, Registration Quality Index, target registration error, brain shift, head-mounted display, real-time monitoring, surgical navigation, real-time