New Real-Time Index Promises to Catch Drifting Virtual Anatomy During Brain Surgery

Neurosurgeons working with mixed reality headsets may soon have a way to know—moment by moment—whether the virtual anatomy floating over their patient’s brain is still telling the truth. Researchers at the University of Salerno have developed and validated a new automated metric, the Registration Quality Index (RQI), that continuously quantifies how well a superimposed virtual anatomical model matches the physical surgical field in real time. Published in the International Journal of Computer Assisted Radiology and Surgery, the study offers a technical proof of concept for closing one of the most stubborn safety gaps in augmented and mixed reality (AR/MR) surgery: the fact that registration accuracy is typically verified once, at the start of a procedure, and then essentially taken on faith for everything that follows.

The stakes could hardly be higher. In neurosurgery, millimetric margins separate eloquent cortex, major blood vessels and white matter tracts from tumour tissue, and prior studies have documented that inadvertent injury to eloquent cortical regions during tumour resection produces permanent neurological deficits in a substantial proportion of patients. Errors exceeding two to three millimetres significantly increase the risk of vascular injury, incomplete resection and neurological deterioration, while deep targets such as the basal ganglia, thalamic nuclei and brainstem demand sub-millimetre precision. In deep brain stimulation surgery, targeting deviations of just one to two millimetres can reduce clinical benefit and provoke stimulation-induced adverse effects. Against these requirements, the errors reported for AR/MR neurosurgical applications—translational errors ranging from 0.62 to 6.93 mm and angular errors from 1.32 to 6.80 degrees depending on hardware and registration method—are sobering.

The problem, the researchers argue, is that the conventional yardsticks of navigation accuracy are fundamentally static. Fiducial Registration Error (FRE), the root mean square distance between corresponding markers after registration, and Target Registration Error (TRE), which measures accuracy at anatomical positions not used in the registration, are both discrete-point measurements computed at a single moment. Neither can capture the dynamic degradation of registration quality during an operation, which can be driven by tracking system drift from thermal effects, electromagnetic interference and line-of-sight occlusions; by patient micro-movements despite rigid cranial fixation; and by brain shift—the displacement of brain tissue caused by cerebrospinal fluid drainage, gravity and tumour resection, which can exceed 10 millimetres. Intraoperative MRI and ultrasound can partially compensate, but these modalities are specialised, interrupt the surgical workflow and are not universally available.

RQI attacks this gap with a deliberately lightweight computer vision pipeline that runs entirely from the headset’s own camera feed. The experimental platform paired a Varjo XR-3 head-mounted display—offering 2880 by 2720 pixels per eye at a 90 Hz refresh rate—with six HTC Vive Lighthouse base stations providing sub-millimetre optical tracking. A ceramic skull phantom, dimensioned to match adult human skull morphometry, served as the anatomical reference, and its virtual counterpart was reconstructed from computed tomography segmentation at 0.5 mm voxel resolution, imported into Unreal Engine 5.0.3 and rendered as a semi-transparent cyan overlay. Spatial registration between the virtual and physical coordinate systems was anchored to a printed 2D ArUco fiducial marker, detected by the headset’s stereo cameras at 30 Hz to compute the six-degree-of-freedom transformation in real time.

The computation itself proceeds in five stages on each passthrough frame. After frame acquisition at 30 Hz and a resolution of 320 by 160 pixels, the skull region is segmented from the background by adaptive thresholding, and Canny edge detection extracts the contours of both the physical phantom and the virtual overlay. Because the virtual model is rendered in cyan—a hue absent from the bone-white phantom—each detected edge pixel can be classified by examining its neighbourhood in the original RGB frame, yielding separate binary maps of physical and virtual boundaries. For every pixel on the physical boundary, the minimum Euclidean distance to the nearest virtual boundary pixel is then computed using OpenCV’s distance transform. Finally, RQI is expressed as the percentage of physical boundary pixels whose displacement exceeds a proximity threshold: a perfectly registered system scores 0 percent, while progressive misregistration drives the value toward 100 percent. Higher RQI, in other words, means worse alignment.

Choosing that proximity threshold was a careful balancing act. A systematic sensitivity analysis across thresholds from 2 to 12 pixels showed that values below 4 pixels produced unstable readings inflated by sub-pixel edge localisation noise, while values above 8 pixels compressed the RQI distribution toward zero and made the metric insensitive to clinically relevant degradation. The selected threshold of 6 pixels—roughly 8 millimetres at the 40 cm working distance, where one pixel corresponds to approximately 1.3 mm—maximised the Pearson correlation with TRE and was independently supported by receiver operating characteristic analysis, discriminating acceptable from degraded registration with a Youden index of 0.87, sensitivity of 0.93 and specificity of 0.94. The authors candidly note that because the threshold was tuned on the same dataset used to characterise the metric, the values carry an optimistic bias and must be confirmed on independent data.

The validation results were striking. Across fifteen experimental trials and seventy-five viewing configurations, RQI correlated strongly with both established metrics: r = 0.89 with FRE (95% CI 0.69–0.96, p < 0.001) and r = 0.93 with TRE (95% CI 0.80–0.98, p < 0.001), the latter meaning that 87 percent of the variance in target accuracy was explained by the surface-wide pixel misalignment. Regression analysis quantified the relationship in practical terms: each 1 percent increase in RQI corresponded to a 0.195 mm increase in FRE and a 0.244 mm increase in TRE. Spearman rank correlations confirmed the relationship was monotonic across the full experimental range—crucial, because for intraoperative monitoring, detecting that registration quality has degraded matters more than knowing the absolute error. Mean FRE was 2.95 ± 0.48 mm and mean TRE 3.78 ± 0.57 mm, while mean RQI was 3.28 ± 1.37 percent.

Speed and usability were also put to the test, with encouraging results. The entire image analysis pipeline averaged about 10 milliseconds per frame on a CPU alongside the GPU-bound rendering, achieving real-time computation at 30 Hz with no dropped frames and no disruption to workflow. A linear mixed effects model across the 75 viewing configurations revealed that viewing angle significantly influenced RQI (p < 0.001, partial η² = 0.49): displacement was lowest at the frontal view and rose at oblique, lateral, posterior and contralateral perspectives, a geometrically predictable effect since errors perpendicular to the viewing axis project maximally. The authors suggest lateral views may serve as a more sensitive early warning of degradation, and that angle-dependent normalisation could improve alert specificity. Three board-certified neurosurgeons rated the system on the System Usability Scale at a mean of 79.0 ± 4.7, well above the acceptability threshold of 68.

The team is equally forthright about limitations. Validation was confined to a rigid phantom under controlled laboratory conditions and cannot account for brain shift, soft tissue deformation, pulsatile brain motion or variable illumination. A particularly important caveat is brain sag—the gravity-driven displacement of the brain after dural opening—which can migrate deep structures by several millimetres while the exposed cortical surface remains comparatively stable; a surface-based metric could then report acceptable alignment even as the true target drifts. The interpretability ranges offered in the study, with band boundaries at 2, 4 and 6 percent RQI, are explicitly exploratory research guidance rather than validated clinical decision boundaries, and the authors stress that RQI is not yet validated for intraoperative clinical decision-making. The most important next step, they write, is a dedicated experiment applying translations and rotations of predetermined magnitude to the registration transform to establish the metric’s sensitivity, linearity, repeatability and failure modes, followed by validation under clinical conditions and integration with intraoperative imaging.

Even so, the implications extend well beyond the cranial theatre. The authors point out that the absence of continuous, marker-free registration monitoring is equally pertinent to spinal navigation, where vertebral mobility between preoperative imaging and the operative position is a recognised source of drift, and where AR/MR guidance and robotic platforms are increasingly used for pedicle screw placement requiring sub-millimetre accuracy. A surface-based stability index computed on exposed bony landmarks could provide intraoperative feedback in those workflows too. If subsequent clinical studies confirm the promise of this proof of concept, the humble act of continuously comparing a cyan wireframe to the anatomy beneath it could become a quiet but essential guardian—watching, frame by frame, that the surgeon’s virtual map never silently stops matching the territory.

Subject of Research: A real-time metric for continuous monitoring of registration stability in mixed reality neurosurgical navigation

Article Title: A real-time metric for quantifying registration stability in mixed reality neurosurgical navigation

Article References: Fontana, C., De Notaris, M., Iaconetta, G., & Cappetti, N. (2026). A real-time metric for quantifying registration stability in mixed reality neurosurgical navigation. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03792-z

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03792-z

Keywords: mixed reality, neurosurgery, neuronavigation, registration accuracy, augmented reality, Registration Quality Index, target registration error, brain shift, head-mounted display, real-time monitoring, surgical navigation, real-time