Visual-inertial odometry combines camera data with inertial measurements to estimate motion where satellite navigation may be unavailable, including indoors, in forests and among buildings. The three estimators represented the same 15-variable navigation state: orientation, position, velocity, gyroscope bias and accelerometer bias. The ESKF propagated uncertainty with first-order error-state equations, whereas the Full UKF used a scaled unscented transform to propagate 31 sigma points through nonlinear motion. The Hybrid retained the ESKF backbone for inertial propagation, velocity, position and visual-position correction, but maintained a separate four-dimensional quaternion-coordinate covariance and nine sigma points for visual-orientation correction. Because the quaternion covariance was not merged with the 15-dimensional ESKF covariance, the architecture did not perform complete fusion of two uncertainty models: it transferred a corrected nominal orientation while preserving ESKF-based translational estimation.

Global and local accuracy produced different winners. Over 30 paired trials on each EuRoC Machine Hall sequence, the Hybrid had the lowest translational ATE on four of five sequences and the lowest rotational ATE on all five. The Full UKF, however, had the lowest one-second translational relative pose error, RPE, on all five, while the ESKF had the lowest one-second rotational RPE on all five. ATE measures disagreement over an entire trajectory after one global rigid alignment and therefore captures accumulated deformation and drift; RPE measures relative-motion consistency over a one-second interval. The result shows that the estimator with the most globally accurate path was not necessarily the one with the sharpest short-term rotational motion. A matched Hybrid-LinearOri ablation used the same Hybrid backbone, observations, seeds and positional correction but replaced the unscented orientation branch with a linearized one. The Hybrid had lower translational ATE, rotational ATE and translational RPE on every EuRoC sequence, while the linearized version retained an advantage in rotational RPE.

Cross-dataset testing qualified the EuRoC result. The same estimator parameters were transferred without retuning to three TUM VI room sequences, with 30 paired trials per sequence. All 360 requested TUM VI runs completed, as did all 600 controlled EuRoC runs. The Hybrid achieved the lowest translational ATE on TUM VI room1 and room3, while the Full UKF led on room5. Rotational and short-horizon relative-motion results did not follow a consistent ordering across the three sequences. Across all eight controlled EuRoC and TUM VI sequences, the Hybrid had the lowest translational ATE in six. The study retained estimator failures as recorded outcomes rather than replacing them with reference states, although none occurred in these controlled benchmarks, and it reported means and sample standard deviations without applying inferential significance tests.

A second experiment replaced reference-derived observations with poses calculated by a common lightweight stereo visual-odometry frontend. The pipeline synchronized and rectified stereo images, extracted up to 2,000 ORB features per rectified image, matched them by Hamming distance, triangulated accepted correspondences and estimated relative motion with PnP-RANSAC. It contained no bundle adjustment, keyframe optimization, loop closure or global pose graph, and failed motion updates were not replaced with ground truth. All four estimators received the same image-derived observations. On EuRoC, the Hybrid had the lowest translational ATE and translational one-second RPE on all five sequences, and the lowest rotational ATE on four. Its translational ATE ranged from 3.726 to 4.297 metres, substantially above its controlled results, while the ESKF ranged from 4.505 to 5.453 metres and the Full UKF from 4.497 to 5.278 metres. Diagnostics showed that this comparison did not imply accurate visual observations: median PnP inlier ratios were 0.880–0.896, yet median frame-to-frame translation-direction errors remained 21.7–23.8 degrees.

The stereo results on TUM VI produced the lowest translational and rotational ATE for the Hybrid on all three room sequences, but exposed large orientation errors shared by every architecture. Depending on the estimator and sequence, rotational ATE ranged from about 73 to 141 degrees. The Hybrid’s translational one-second RPE was lowest on room3 and room5, yet it was not lowest for rotational RPE on any of the three sequences. These large values mean the TUM VI image experiment should be interpreted as a comparison of how the backends respond to a common, imperfect observation stream, not as evidence that any estimator achieved high absolute orientation accuracy. The discrepancy between controlled and image-derived results is central to the study: backend differences remained detectable with real images, but each final trajectory also reflected errors accumulated by the sequential visual frontend.

Two sensitivity studies separated observation quality from the confidence assigned to observations. In the first, injected position noise ranged from 0.075 to 0.30 metres and orientation noise from 0.25 to 1.00 degree, while estimator-assumed uncertainty remained fixed; all 5,400 runs completed. Among 45 sequence-noise conditions, the Hybrid had the lowest translational ATE in 31 and the lowest rotational ATE in all 45. The Full UKF led translational RPE in 30 conditions, and the ESKF led rotational RPE in 29. Across the tested position-noise range, the Hybrid’s mean translational RPE rose from approximately 0.072 to 0.226 metres; across the orientation-noise range, its mean rotational RPE increased from about 0.757 to 2.485 degrees. In the complementary analysis, stereo observations were held fixed while the estimators’ assumed position and orientation uncertainties were varied. The Hybrid then had the lowest translational ATE in 44 of 45 conditions and the lowest translational RPE in all 45, while rotational outcomes were more evenly divided.

Dedicated timing measurements showed the principal computational distinction among the backends. Across 1,346,350 measured updates per estimator, mean processing time was 0.005700 milliseconds for the ESKF, 0.017855 milliseconds for the Full UKF, 0.005533 milliseconds for Hybrid-LinearOri and 0.005371 milliseconds for the Hybrid. The Full UKF therefore required about 3.32 times the Hybrid’s mean backend time. At the 99th percentile, the respective times were 0.0230, 0.0647, 0.0215 and 0.0207 milliseconds; the Hybrid also had the lowest 95th-percentile time, at 0.007646 milliseconds, and the lowest mean. The timing pattern was consistent with the architectures: full-state unscented propagation required 31 points, whereas the Hybrid used nine points only during orientation correction. However, the measurements covered estimator backends, not image loading, feature extraction, stereo matching, triangulation, PnP-RANSAC, file operations or logging. They did not establish end-to-end latency, processor or memory demand, energy use, hard-real-time behavior or feasibility on an embedded UAV computer.

The findings support a targeted rather than universal advantage for nonlinear filtering. Within the tested implementations, observations and sequences, selective unscented orientation correction was associated with lower accumulated trajectory error without the typical backend cost of full-state sigma-point propagation. The benefit remained conditional on the metric, sequence and visual-observation quality. The Hybrid was not uniformly more accurate, and the ablation compared complete correction stages that also differed in covariance treatment, rather than isolating the unscented transform alone. The study used selected indoor EuRoC and TUM VI sequences, Gaussian perturbations, fixed covariance settings and a lightweight visual frontend. Outdoor flight, aggressive motion, illumination changes, motion blur, visually degraded scenes, non-Gaussian uncertainty, advanced frontends and onboard hardware remained untested. The supported conclusion is that selective unscented processing can improve global trajectory accuracy in these backends while avoiding the measured mean backend cost of full-state propagation, not that the Hybrid is a universally superior substitute for error-state or full-state filtering.

Subject of Research: Accuracy and estimator-backend processing trade-offs among ESKF, Full UKF and hybrid ESKF–UKF visual-inertial estimators

Article Title: Accuracy-efficiency trade-offs in visual-inertial odometry backends: Comparative evaluation of ESKF, UKF, and hybrid ESKF-UKF using real stereo-inertial data

Article References: Nigusie, A. T., Lin, B., & Teshome, F. B. (2026). Accuracy-efficiency trade-offs in visual-inertial odometry backends: Comparative evaluation of ESKF, UKF, and hybrid ESKF-UKF using real stereo-inertial data. Results in Engineering, 32, Article 113487. https://doi.org/10.1016/j.rineng.2026.113487

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113487

Keywords: visual-inertial odometry, error-state Kalman filter, unscented Kalman filter, hybrid filtering, trajectory accuracy, sigma-point propagation, stereo vision, robot navigation, EuRoC MAV, TUM VI, Accuracy-efficiency, trade-offs

The suite, called ParaStereoSync and KineViz-3D, is distributed under the MIT license and runs on Windows, Linux, and macOS using freely available Python libraries including OpenCV, NumPy, SciPy, Matplotlib, and Tkinter. It is organized as five standalone graphical applications linked by CSV files and video streams. Three utility tools, VidTrim, tkCalib2, and TempArUco, prepare the inputs; ParaStereoSync performs the tracking and three-dimensional reconstruction; and KineViz-3D turns the resulting coordinate histories into interactive spatial and kinematic visualizations. No commercial software, hardware dongle, or paid cloud service is required at any stage of the pipeline.

The team’s motivation stems from a well-known mismatch between existing measurement tools and the realities of large-scale testing. Digital image correlation, or DIC, has become the dominant vision-based technique in structural engineering, tracking small patches of a speckle-coated surface between images to recover full displacement fields. Mature packages such as Ncorr, OpenCorr, py2DIC, D²IC, and commercial systems like GOM ARAMIS and VIC-3D have made the method widely accessible. Yet, the authors argue, these tools were all designed for a specific experimental setting: a densely painted surface, uniform controlled illumination, high image resolution, small displacement ranges, and hardware-synchronized cameras. None of those conditions reliably holds on a full-scale ceiling system in a test hall, where speckling an entire bay is impractical, lighting varies, and consumer cameras run on independent internal clocks.

That last problem, camera synchronization, is the suite’s headline technical innovation. When two cameras lack a shared trigger signal, they record on separate clocks and accumulate temporal offsets, so each camera effectively sees a slightly different moment in time. In stereo photogrammetry, the resulting three-dimensional coordinates contain spurious apparent displacements proportional to the structure’s velocity multiplied by the timing error. ParaStereoSync attacks this in software. Its synchronization algorithm tests a range of candidate frame lags between the two videos, interpolating the second camera’s coordinates at each lag, triangulating the matched points, and computing the mean-square reprojection error. The lag producing the lowest error is selected, and a cubic spline fit around the minimum refines the result to sub-frame precision. Crucially, a separate lag is estimated for each tracked point, which absorbs point-specific timing offsets including those caused by the rolling-shutter readout of CMOS sensors, without requiring any explicit sensor timing calibration or special hardware.

Tracking itself relies on the Enhanced Correlation Coefficient criterion under an affine warp model, which is more robust to changes in illumination than simpler sum-of-squared-differences approaches. A multi-step initialization propagates the converged transform from each frame into the next, restricting the search to a localized region and allowing the tracker to follow the large inter-frame displacements that occur during strong-motion excitation. If convergence fails under the affine model, the tracker automatically retries with a simpler Euclidean model, then a pure translation model, with progressively heavier Gaussian smoothing at each retry level. Rather than tracking every pixel the way full-field DIC does, the software follows sparse targets at structurally meaningful locations: printed ArUco fiducial markers, or even natural features such as bolts, connection hardware, and panel corners, eliminating the need for speckle preparation entirely.

Speed comes from camera-level parallelism. Both cameras are dispatched as independent worker processes through Python’s ProcessPoolExecutor, each running the full tracking pipeline on its own video file. The authors deliberately rejected point-level parallelism, noting that video decoding is input-output-bound and many small processes reading the same file would saturate the storage bus. On an Intel Core i7-11800H laptop, the approach achieved speedups of 1.44 to 1.74 times. In their demonstration, tracking 322 ceiling panel points across 2,300 frames took 14.63 minutes sequentially and 8.42 minutes in parallel, with a 100 percent tracking success rate. Tracking 64 T-bar grid intersections over the same frames took 1.95 minutes with parallelism versus 3 minutes without.

The utility tools tackle the setup bottlenecks that historically made high-node-count campaigns tedious. VidTrim wraps FFmpeg in a lightweight graphical interface and trims raw footage losslessly using stream-copy mode, so no frames are re-encoded or degraded; it also offers a motion preview that lets users confirm a target is trackable before committing to a full run. tkCalib2 supports two calibration methods: a standard multi-image checkerboard procedure, and a single-image approach that solves camera pose from known three-dimensional control points, valuable when a specimen is too large for a big calibration board. TempArUco is perhaps the biggest time-saver: it reads the first frame from each camera, enhances contrast with CLAHE preprocessing, detects all ArUco markers, matches them across cameras by their unique binary IDs, and writes template files automatically. In the team’s demonstration, it produced templates for 322 markers in roughly 90 seconds, replacing what would otherwise have been a two-hour interactive clicking session.

The pipeline was put to the test at the Carleton University seismic resilience laboratory on a suspended lay-in ceiling system spanning 5.156 by 5.156 meters, comprising 81 panels on a T-bar grid mounted on a purpose-built shaking table. One-hundred-millimeter ArUco markers sat at every panel corner and 50-millimeter black-and-white markers at every grid intersection, yielding 324 and 64 targets. Two GoPro HERO12 Black cameras recorded at 4K resolution and 60 frames per second from a stereo baseline of 2.08 meters at an average standoff of 3.87 meters. Synchronization analysis found a well-defined minimum at 27 frames, refined to an average of 27.22 frames, with per-node lags ranging from 26.60 to 28.88. Validation against accelerometer-derived displacements in prior large-scale experiments showed a root-mean-square error of at most 2.970 millimeters and a mean absolute error below 1.708 millimeters across all configurations.

KineViz-3D then turns the raw tracking output into engineering insight. Its centerpiece is an animated three-dimensional heatmap of displacement interpolated onto a 50 by 50 grid, with colormap bounds fixed to global extremes so amplitudes remain comparable across frames. A six-panel profile evolution dashboard synchronizes out-of-plane and in-plane displacement profiles, three phase-space orbit plots, and a live bar chart of instantaneous displacement magnitude, and it automatically annotates the frame and node pair with the maximum displacement differential, a direct quantitative indicator of emerging connection failure. Selecting four nodes on a rigid body invokes the Kabsch algorithm to decompose six-degree-of-freedom motion. In the ceiling test, the analysis identified a 4.91-millimeter displacement differential between outermost and innermost grid nodes, and the rigid-body decomposition revealed a 0.3-degree panel rotation about the Y axis, later confirmed by visual inspection of a loosened grid connection.

The implications extend well beyond ceiling research. Any experiment using two or more unsynchronized consumer cameras to observe dynamic phenomena faces the same inter-camera timing problem, and the reprojection-error synchronization approach applies wherever stereo pairs can see identifiable features in motion, from biomechanics to geotechnical centrifuge testing to field bridge monitoring. The modular design lets researchers feed data from commercial systems directly into KineViz-3D, or use ParaStereoSync as a standalone stereo photogrammetry pipeline. The authors identify vectorized, batched triangulation with optional GPU offload as the path to scaling to higher node densities. For now, the message to cash-strapped laboratories is striking: a laptop, two action cameras, and a sheet of printed markers can now capture three-dimensional structural motion across hundreds of points, at a fraction of the cost of equivalent commercial instrumentation.

Subject of Research: Open-source stereo-vision software for tracking 3D structural displacements in earthquake engineering experiments

Article Title: ParaStereoSync and KineViz-3D: A Python suite for parallelized stereo-vision tracking and spatio-temporal visualization of structural displacements

Article References: Seyfu, M. K., Yang, Y.-S., Flude, C. C. W., Lau, D. T., & Erochko, J. (2026). ParaStereoSync and KineViz-3D: A Python suite for parallelized stereo-vision tracking and spatio-temporal visualization of structural displacements. SoftwareX, 36, Article 103139. https://doi.org/10.1016/j.softx.2026.103139

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103139

Keywords: stereo vision, structural engineering, earthquake engineering, digital image correlation, camera synchronization, shake-table testing, open-source software, Python, suspended ceilings, displacement measurement, ArUco markers, computer vision

Developed by Jan Hejda and colleagues and described in the journal SoftwareX, BatPose is a Python package that carries a movement recording from raw stereo video all the way to biomechanical statistics. It estimates three-dimensional human pose in real physical units, computes nine joint and trunk angles, and reports range of motion, variability, and left-right symmetry, all through a graphical interface or command-line tools that run entirely on a local CPU. The software is released under the GNU General Public License, with version 1.1.0 available on GitHub, and the team has published a preliminary validation against a professional marker-based system.

The technical pipeline follows a classical stereo-geometry route rather than an end-to-end learned model. First, the user waves a ChArUco calibration board, a checkerboard embedded with fiducial markers, in front of two cameras. The software detects the board across sampled frames, deliberately keeping views that cover different regions of the image, and then solves each camera’s internal parameters and the relative position between the two cameras using Zhang’s planar calibration method. Three lens distortion models are supported, from a standard pinhole model to wide-angle and fisheye formulations, and the program warns users when the chosen model does not match the physical lens, since a mismatch can silently warp every reconstructed coordinate.

Calibration quality is graded automatically using the root mean square reprojection error: below 0.5 pixels is rated excellent, up to 1.5 pixels is acceptable, and anything higher triggers a recommendation to recalibrate, often because the lens is wider than the model allows. The developers are careful to note that this score is an in-sample statistic computed on the calibration images themselves, not a validated bound on three-dimensional accuracy at the distance where movement is later recorded. That kind of methodological honesty runs throughout the project, which distinguishes it from many flashy pose-estimation demos.

Once the cameras are calibrated, a two-dimensional pose detector locates seventeen body keypoints in each view. The default backend is MediaPipe BlazePose, with RTMPose available as an alternative, and both outputs are mapped onto the standard COCO-17 skeleton convention. The software then triangulates each keypoint in three dimensions using the direct linear transform, the workhorse algorithm of multi-view geometry. A confidence gate rejects joints whose detections fall below a threshold in either view or whose reprojection error is too large, and a One Euro filter smooths each coordinate over time with settings that users can tune to match their frame rate and the speed of the movement being studied.

A particularly thoughtful feature is the floor world frame. By triangulating the corners of a board lying on the ground and aligning them with the Kabsch algorithm, BatPose establishes an upright coordinate frame with its origin at the board center, expressed in meters. Every recording from every session then shares the same spatial reference, which makes cross-session comparisons meaningful. From the reconstructed skeletons, the package computes bilateral knee, hip, elbow, and shoulder angles as true three-dimensional geometric segment angles, plus trunk inclination relative to vertical, and derives statistics including peak angular velocity and a bilateral symmetry index originally developed for force-platform gait analysis.

The validation against a twelve-camera Qualisys marker-based system at 300 Hz gives a first quantitative picture of what to expect. Three participants performed arm raises, biceps curls, marching, forward bends, and squats under three different stereo camera configurations. Across eight flash-synchronized recordings, the frame-weighted mean per-joint position error was 52.1 millimeters, with a median of 42.1 millimeters, and 77.3 percent of joint observations met the criteria for direct measurement. Mean absolute error for the nine angle outputs averaged 7.50 degrees, with concordance correlation coefficients between 0.910 and 0.968. Notably, performance depended strongly on camera geometry: one configuration with a wider baseline and steeper convergence angle achieved errors around 45 millimeters and over 92 percent coverage, while another dropped to under 50 percent coverage.

The team also compared angle outputs against the Vicon-derived TotalCapture dataset, finding a mean recording-level error of 9.90 degrees across five participants, though with wide limits of agreement and coverage as low as 44.6 percent in some recordings. Repeatability tests showed that repeated calibrations of a fixed rig agreed to within a few millimeters of baseline and a fraction of a degree of rotation, and that repeated detections on identical input reproduced coordinates almost exactly, with occasional large discrepancies near tracking gaps. The authors repeatedly emphasize that these are preliminary technical validations under specific tested conditions, not evidence of clinical validity or generalizability to other populations, tasks, or camera arrangements.

Those caveats matter, and the limitations section is refreshingly candid. The software matches people between the two camera views by detection order rather than by identity, which means it is designed for a single subject and may fail when multiple people cross paths. Accuracy is ultimately bounded by the quality of the two-dimensional detector and the calibration, and the system reconstructs kinematics only, not the musculoskeletal forces that pipelines built on OpenSim provide. Clothing, occlusion, and camera distance were not systematically varied in controlled experiments, and derived statistics such as peak velocity and symmetry indices cannot be assumed reliable from joint-angle agreement alone. The developers plan to add cross-view identity matching and additional pose backends in future releases.

Even with those boundaries, the implications are striking. A rehabilitation clinic, a sports team, or an ergonomics laboratory in a low-resource setting could now quantify movement asymmetries, track recovery across weeks, or analyze exercise technique with equipment costing a tiny fraction of a professional capture suite, while keeping all recordings on institutional computers rather than uploading them to a cloud service. Cached intermediate files mean users can swap detectors without recalibrating and retune reconstruction without re-running detection, and a built-in camera simulator lets students learn the workflow without any hardware at all. In a field where the gap between laboratory-grade measurement and everyday practice has persisted for decades, a compact, transparent, fully local stereo pipeline with published validation numbers is a genuinely meaningful step toward democratizing the science of human movement.

Subject of Research: Markerless stereo 3D human pose estimation and biomechanical analysis software

Article Title: BatPose: Single-subject stereo 3D human pose estimation and biomechanical analysis from synchronized cameras

Article References: Hejda, J., Sokol, M., Fischer, A. G., Radus, L. F., Leová, L., Volf, P., & Kutílek, P. (2026). BatPose: Single-subject stereo 3D human pose estimation and biomechanical analysis from synchronized cameras. SoftwareX, 36, Article 103093. https://doi.org/10.1016/j.softx.2026.103093

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103093

Keywords: BatPose, motion capture, markerless pose estimation, biomechanics, stereo vision, camera calibration, MediaPipe BlazePose, triangulation, joint angles, rehabilitation, open-source software, Qualisys validation