Electromagnetic tracking brings sub-millimeter augmented reality to robotic surgery

Robot-assisted surgery offers clear clinical advantages, but it remains limited by reduced visualization and the lack of tactile feedback. A study published in the International Journal of Computer Assisted Radiology and Surgery reports a technical step toward addressing this: a calibration and tracking pipeline that overlays preoperative imaging onto the live laparoscopic video of a robotic endoscope, using an externally mounted electromagnetic tracker to keep the virtual and real images aligned in real time.

The central technical challenge is knowing, at every instant, exactly where the endoscope camera is and how it is oriented relative to the patient’s anatomy. The team’s approach attaches a rigid electromagnetic sensor to the robotic endoscope, allowing the system to continuously determine the camera’s position and pose. Before the tracker can be trusted, however, the camera itself must be calibrated: the system needs the precise relationship between what the sensor reports and what the lens actually images. The researchers solved this by rigidly mounting a second sensor to a calibration checkerboard and acquiring 35 calibration images.

A key question in any calibration protocol is how many images are actually needed. More frames generally improve accuracy, but each additional acquisition adds setup time in an operating room. To answer this quantitatively, the team drew 2600 random subsets containing between 5 and 30 images, calibrated each subset using RANSAC-based robust estimation, and evaluated the results against 16 separately recorded checkerboard frames using three-dimensional Euclidean error. Accuracy and robustness improved with increasing dataset size, and 19 images was the smallest dataset size whose median 3D error fell within 5 percent of the best observed performance. That configuration yielded a mean 3D Euclidean error of 0.541 plus or minus 0.060 millimeters.

Laboratory accuracy is one thing; operating room reality is another. Robotic endoscopes are handled and repositioned throughout a procedure, so any tracking hardware must survive clinical handling without degrading the calibration. The researchers therefore designed a clinically suitable adapter carrying the electromagnetic tracker and rigidly mounted it on the robotic endoscope. When evaluated with the same calibration protocol, the clinical adapter achieved a mean 3D Euclidean error of 0.784 plus or minus 0.075 millimeters, still sub-millimeter but measurably higher than the benchtop configuration, reflecting the practical tolerances introduced by a mountable clinical device.

The decisive test came in the operating room. The system was deployed during 16 robot-assisted pelvic lymph node dissection procedures. According to the study, the setup provided stable, real-time tracking and augmented reality visualization throughout these procedures, without requiring substantial modification of the standard robotic workflow. The externally mounted tracker allowed continuous endoscope tracking as the camera moved, which is essential if the overlay is to remain registered during the constant repositioning that characterizes robotic surgery.

The significance of sub-millimeter accuracy lies in what it enables technically. An augmented reality overlay is only useful if the projected image stays registered to the underlying anatomy as the endoscope moves. By demonstrating that both the calibration dataset and the clinical adapter can hold mean errors well under one millimeter, the study addresses one of the main quantitative requirements for intraoperative image overlay on robotic platforms.

The authors position the work as a technical foundation rather than a finished clinical product. The reported results establish that the calibration and tracking approach can be integrated into a clinical robotic workflow and support real-time visualization, but the study evaluated system performance and feasibility during the procedures rather than measuring whether the overlays changed surgical decisions or patient outcomes. Further development and validation of image-guided augmented reality applications in robot-assisted surgery would be needed before such a system could influence patient care.

Even so, the findings mark a notable milestone in image-guided robotic surgery. The combination of a compact 19-image calibration protocol, a rigidly mounted electromagnetic tracker, and demonstrated sub-millimeter overlay accuracy across 16 real procedures shows that augmented reality navigation can be integrated into the clinical robotic workflow while allowing continuous tracking of the endoscope through an externally mounted sensor.

Subject of Research: Camera calibration and electromagnetic tracking for augmented reality navigation in robot-assisted surgery

Article Title: Camera calibration and tracking for augmented reality during robotic surgery

Article References: Venix, L. P. J., Heerink, W. J., Hiep, M. A. J., Saiz, L. A., Kerkhof, E., Ruers, T. J. M., & Fusaglia, M. (2026). Camera calibration and tracking for augmented reality during robotic surgery. International Journal of Computer Assisted Radiology and Surgery. https://doi.org/10.1007/s11548-026-03809-7

Image Credits: AI Generated

DOI: 10.1007/s11548-026-03809-7

Keywords: robotic surgery, augmented reality, electromagnetic tracking, camera calibration, image-guided surgery, laparoscopy, surgical navigation, RPLND, medical imaging, computer-assisted surgery, Camera, calibration