When Accurate Tracking Isn’t Enough: New Score Ranks AI Motion Systems by Biomarker Reliability

Artificial intelligence can now watch a mouse reach for a food pellet and, without any physical markers, trace the exact path of its paw, wrist, elbow and shoulder frame by frame. This capability, known as markerless pose estimation, has transformed quantitative movement analysis in neuroscience, allowing researchers to extract detailed kinematic trajectories from ordinary video recordings. But a new study published in Frontiers in Neuroinformatics argues that the way scientists evaluate these AI tracking systems is fundamentally incomplete, and it proposes a solution that could change how movement biomarkers are validated across preclinical and clinical research.

The research, led by Valentin Fernandez and colleagues at INSERM U1084 and the University of Poitiers, together with collaborators in Coimbra and Samarkand, addresses a subtle but consequential blind spot. Current evaluation of pose estimation pipelines relies almost exclusively on frame-wise spatial accuracy metrics such as root mean square error (RMSE), percentage of correct keypoints (PCK) and mean average precision (mAP). These metrics measure how closely the AI’s predicted landmark positions match human annotations in individual frames. What they fail to capture, the authors contend, is whether the resulting trajectories are reliable enough to yield trustworthy kinematic biomarkers downstream.

The problem becomes acute when those biomarkers involve derivatives. Quantities such as velocity, acceleration and jerk, the rate of change of acceleration, are computed by temporal differentiation of the reconstructed trajectories. Differentiation acts as a noise amplifier: high-frequency tracking jitter and frame-to-frame fluctuations that barely register in a spatial accuracy score can be magnified into substantial distortions of the derived movement features. Two pipelines with nearly identical localisation accuracy may therefore produce trajectories of very different quality once temporal consistency, occlusion handling and kinematic fidelity are taken into account.

To close this gap, the team developed a reliability-centred evaluation framework built around a new composite metric, the Motor Biomarker Reliability Composite Score, or MBRCS. The score integrates four complementary dimensions of trajectory reliability: spatial accuracy, temporal stability, occlusion robustness and kinematic fidelity. Each component is normalised to a scale from zero to one using robust percentile-based scaling, in which values are clipped to the fifth and ninety-fifth percentiles of the evaluated distribution to reduce sensitivity to outliers. Higher values indicate greater trajectory reliability. The authors emphasise that a normalised score near zero reflects relative positioning within the benchmark, not an absence of capability.

The weighting scheme assigns greater importance to temporal stability and kinematic fidelity, a choice made a priori because derivative-based kinematic descriptors are particularly vulnerable to temporal fluctuations and noise amplification. Sensitivity analyses using alternative weighting configurations, including uniform weighting and accuracy-prioritised schemes, produced highly consistent architecture rankings with Spearman correlation coefficients above 0.9, indicating that the composite score is not strongly dependent on the precise weights chosen. The MBRCS is intended as a comparative benchmarking tool for selecting pipelines, not as an absolute measure of performance, and computational efficiency was deliberately excluded from the score.

The framework was tested on a demanding experimental paradigm: the Montoya staircase task, in which mice retrieve food pellets from descending staircases on either side of a central platform. Adult C57BL/6 mice, twelve control animals and twelve with unilateral lesions of the primary motor cortex, were recorded from a lateral view at 120 frames per second with a resolution of 1,280 by 1,024 pixels. Around 4,800 frames from 24 behavioural recordings were manually annotated, covering landmarks including the shoulder, elbow, wrist, forearm, base of the fingers and fingertips. The task is a particularly rigorous benchmark because skilled reaching involves small-amplitude distal limb displacements, rapid temporal transitions and frequent partial occlusions caused by self-occlusion and contact with the apparatus.

Nine DeepLabCut-based convolutional neural network architectures were trained and tested under identical conditions using DeepLabCut version 3.0 with a PyTorch backend, spanning ResNet-50, ResNet-101 and ResNet-152, MobileNetV2, EfficientNet-B0 and EfficientNet-B3, and several DLCRNet-based configurations. All models were trained for 600,000 iterations on the same annotated dataset with the same split, optimisation pipeline and post-processing, so that performance differences could be attributed to architecture rather than training strategy. Predicted landmarks with a probability below 0.9 were treated as low-confidence detections; short gaps of up to five consecutive frames were reconstructed by linear interpolation, while longer interruptions, which accounted for less than 3 percent of low-confidence segments, were excluded from kinematic analysis. Trajectories were then smoothed with a Gaussian filter of sigma equal to 2 frames.

The results revealed a striking dissociation. Spatial localisation metrics remained comparatively clustered across architectures, with RMSE ranging only from 2.80 to 3.58 pixels, PCK at a 5-pixel threshold from 90.70 to 95.90 percent, and mAP from 85.63 to 91.07 percent. Yet the reliability dimensions told a very different story. Temporal stability varied substantially, and normalised jerk values, the downstream biomarker used to assess kinematic fidelity, ranged from 1.60 for the best-performing DLCRNet_ms5 to 3.85 for MobileNetV2. When all four dimensions were combined, MBRCS values spanned almost the entire normalised range, from 0.00 to 0.99, with DLCRNet_ms5 at the top and MobileNetV2 at the bottom.

Crucially, these reliability differences had functional consequences. In an external validation step deliberately excluded from the MBRCS itself, the researchers assessed whether trajectories with higher intrinsic reliability better preserved biologically meaningful differences between the control and lesion groups. Using Cohen’s d effect sizes as the primary measure of known-group validity, complemented by receiver operating characteristic analysis with logistic regression classifiers under stratified five-fold cross-validation, they found that architectures with higher temporal stability and kinematic fidelity showed greater separation of jerk distributions between groups. DLCRNet_ms5 and DLCRNet_stride32 achieved both larger effect sizes and higher AUC values, whereas MobileNetV2 showed the weakest discriminative performance, with an AUC of approximately 0.60.

The authors are careful to frame their contribution as complementary rather than substitutive. Spatial localisation accuracy remains meaningful, and the best-performing architectures in the composite score were also the most accurate localisers, but the additional dimensions exposed differences that frame-wise metrics alone could not reveal. The framework’s limitations are acknowledged: the evaluation was restricted to DeepLabCut-based architectures and a single downstream biomarker, robustness was assessed primarily under occlusion-related perturbations rather than motion blur or reduced frame rate, and variability from independent training repetitions was not evaluated. Nevertheless, the researchers argue that the methodology offers a structured, reproducible path toward benchmarking pose estimation pipelines by the quality of the kinematic information they preserve, a shift with particular relevance for studies of subtle motor impairment, longitudinal monitoring and acquisition conditions plagued by partial occlusions, where the reliability of the trajectory, not just the accuracy of the pose, determines whether a biomarker can be trusted.

Subject of Research: A reliability-centred evaluation framework for markerless pose estimation in mouse movement analysis

Article Title: From pose estimation accuracy to motor biomarker reliability: a reliability-centred evaluation framework for markerless movement analysis

Article References: Fernandez, V., Abdiyeva, K., Brot, S., Perreira de Almeida, L., Gaillard, A., & Fernandez-Maloigne, C. (2026). From pose estimation accuracy to motor biomarker reliability: a reliability-centred evaluation framework for markerless movement analysis. Frontiers in Neuroinformatics, 20, Article 1790241. https://doi.org/10.3389/fninf.2026.1790241

Image Credits: AI Generated

DOI: 10.3389/fninf.2026.1790241

Keywords: markerless pose estimation, DeepLabCut, motor biomarkers, kinematic analysis, trajectory reliability, Montoya staircase task, Normalized Jerk Score, occlusion robustness, temporal stability, neuroinformatics, movement smoothness, convolutional neural networks