A unified perception framework combining optimized neural network detection, gradient-based sequential adaptation, and dynamic multi-object tracking has demonstrated measurable improvements in identifying vulnerable road users while preserving previously acquired visual representations across disparate operational domains. Developed by researchers Faryal Aurooj Nasir and Salman Liaquat, the YOLOv8-Dynamic-SORT architecture directly tackles the tripartite challenge confronting autonomous driving perception: delivering high detection accuracy, sustaining real-time processing speeds onboard moving vehicles, and adapting to newly encountered environments without experiencing catastrophic forgetting. Modern self-driving vehicles must reliably recognize pedestrians, cyclists, and other non-motorized participants who lack physical protection, yet conventional perception models are typically trained once on static benchmarks. When deployed into novel geographic regions or shifting lighting conditions, static networks frequently suffer severe performance degradation. The study published in Discover Artificial Intelligence shows that carefully calibrating optimization dynamics and switching tracking algorithms at runtime according to local scene complexity allows automated perception systems to maintain reliable identity tracking and domain adaptability.
To establish a high-performing baseline for the detection module, the researchers conducted a rigorous comparative evaluation encompassing one representative two-stage detector, Faster R-CNN, alongside three generations of single-stage architectures, including small and extra-large variants of YOLOv5, YOLOv7, and YOLOv8. Experiments were carried out on the VisDrone2019 dataset, which features challenging aerial camera views, dense crowds, small object scales, and complex urban backgrounds. For the vulnerable road user detection task, annotations were consolidated into four primary target classes: pedestrian, people, bicycle, and tricycle, with motorized vehicles filtered out. Under uniform training and evaluation settings using an intersection over union threshold of 0.50 and a confidence threshold of 0.20, YOLOv8x achieved the most favorable balance of detection accuracy and computational speed. The model recorded a precision of 0.763, an mAP@0.50 of 0.541, an F1-score of 0.593, and an inference latency of 9.9 milliseconds per image, allowing high-throughput processing well above practical operating demands.
Although Faster R-CNN attained higher sensitivity across the test split, achieving a recall of 0.670 and an F1-score of 0.604 compared to 0.485 recall for YOLOv8x, its computational demands proved prohibitive for onboard vehicular platforms. Faster R-CNN required an average inference time of 220 milliseconds per image, translating to a processing rate of approximately 4.55 frames per second. The study established an explicit Pareto admissibility threshold requiring candidate architectures to sustain at least 25 frames per second on the testing workstation. Because collision avoidance mechanisms and motion planning systems require continuous, low-latency sensory input, Faster R-CNN was ruled inadmissible despite its superior recovery of annotated bounding boxes. Selecting YOLOv8x accepted an explicit recall penalty, missing over half of the annotated targets in dense scenes, yet provided the real-time responsiveness necessary to enable subsequent sequential domain adaptation and temporal tracking modules within realistic computational limits.
Having isolated YOLOv8x as the foundational detector, the authors explored how sequential fine-tuning influences the preservation of previously mastered visual features when an autonomous platform enters a novel environment. In standard continual learning scenarios, exposing a neural network to new data distributions causes internal weights to shift, overwriting earlier representations through catastrophic forgetting. While dedicated continual learning frameworks often employ complex parameter regularization penalties, generative replay buffers, or dynamic architectural expansions, these techniques impose substantial memory and computing burdens on embedded automotive processors. The authors instead investigated gradient-based sequential adaptation, assessing whether the intrinsic mathematical properties of the optimization algorithm alone could govern the stability-plasticity balance during transfer learning without modifying network parameters, reserving replay memory, or introducing supplementary loss penalties into the objective function.
The experimental protocol tested domain transfer by adapting the VisDrone2019-trained YOLOv8x detector to the Caltech Pedestrian Dataset, which provides vehicle-mounted urban street imagery recorded at 640 by 480 pixel resolution. This transition imposed a severe domain shift characterized by changes in camera perspective, image resolution, pedestrian scale, and motion blur, moving from aerial surveillance to ground-level forward-facing vehicle perspectives. Unlike VisDrone, the Caltech dataset provides annotations strictly for pedestrians, meaning the experiment tested single-class visual adaptation rather than vocabulary expansion. Adaptation configurations were evaluated using three independent random seeds to quantify run-to-run variability. Standard Stochastic Gradient Descent was contrasted against the adaptive Adam optimizer under an identical base learning rate of 0.01, isolating the empirical influence of the update mechanisms on knowledge retention across both visual benchmarks.
Across the multi-seed evaluation, Stochastic Gradient Descent demonstrated a marked superiority in both target-domain acquisition and source-domain knowledge retention. When adapted using Adam, YOLOv8x attained a Caltech mAP@0.50 of 0.461 plus or minus 0.03, while exhibiting severe source degradation when evaluated again on the original VisDrone2019 test set. Adam adaptation resulted in an absolute forgetting measure of 0.097 plus or minus 0.007, translating to an 18.04 percent relative performance drop from its initial 0.539 mAP level, along with a backward transfer score of minus 0.097. Conversely, fine-tuning with Stochastic Gradient Descent produced an improved Caltech mAP@0.50 of 0.608 plus or minus 0.03, while restricting absolute forgetting on VisDrone to 0.021 plus or minus 0.003, representing a relative loss of only 3.95 percent and a backward transfer of minus 0.021.
The authors analyzed the theoretical dynamics underlying this divergent retention behavior, identifying optimizer mechanics as a key factor. Adam calculates adaptive per-parameter step sizes based on running estimates of first and second gradient moments. During fine-tuning on a novel domain, parameters that experienced minimal gradient activity during initial source training are heavily scaled upward by Adam, driving rapid displacement along directional subspaces that encode original feature representations. This displacement harms source task retention without necessarily benefiting target accuracy. By contrast, Stochastic Gradient Descent enforces uniform step sizing combined with mini-batch stochasticity, producing smaller directional deviations in low-gradient subspaces. This operates as an implicit regularization mechanism, allowing the network to assimilate ground-level pedestrian patterns smoothly while retaining aerial detection features. However, the authors noted that maintaining a shared nominal learning rate of 0.01 did not equalize effective update steps, leaving learning rate sensitivity as an open experimental variable.
Beyond spatial localization and continual adaptation, autonomous vehicle safety depends fundamentally on continuous temporal association to project future trajectories and preserve object identities through occlusions. Conventional tracking-by-detection systems deploy a static tracking algorithm throughout operational lifecycles, forcing engineers to accept rigid trade-offs between tracking precision and execution overhead. High-accuracy appearance-based trackers integrate deep re-identification networks that extract visual embeddings, maintaining robust identity continuity across extended occlusions but incurring heavy computational penalties. In contrast, lightweight motion-centric trackers rely exclusively on spatial overlap and Kalman filter trajectory forecasts, maximizing throughput in uncluttered traffic but degrading rapidly in crowded, highly occluded crossings where track fragmentation and identity switches proliferate.
To overcome the operational shortcomings of static trackers, the researchers engineered Dynamic-SORT, an adaptive tracking module that arbitrates among five distinct BoxMOT algorithms at runtime based on measurable scene characteristics. The integrated tracking portfolio comprises BoTSORT, DeepOCSORT, OCSORT, ByteTrack, and StrongSORT. Dynamic-SORT continuously analyzes input frames by calculating three normalized visual descriptors: crowd density, defined as the number of detected road users per unit frame area; average object velocity, computed from centroid displacements across consecutive video frames at the network input resolution; and occlusion ratio, representing the fraction of detections sharing substantial intersection-over-union overlap exceeding 0.50. Evaluated sequentially on a fixed 30-frame decision interval, these descriptors trigger deterministic tracker assignments mapped to the prevailing environmental complexity.
The deterministic switching hierarchy operates according to five fixed validation thresholds. When end-to-end processing times exceed the platform latency budget of 40 milliseconds, the module invokes a latency override that defaults immediately to ByteTrack to safeguard real-time throughput. In complex scenes where crowd density exceeds 0.050 or occlusion surpasses 0.30, the framework activates BoTSORT to utilize combined motion and appearance features. If average target velocity exceeds 5.0 pixels per frame, DeepOCSORT is selected to handle nonlinear motion dynamics. Moderate crowd density above 0.020 or occlusion above 0.08 routes processing to StrongSORT, while sparse, low-motion environments revert to ByteTrack. Crucially, when switching tracking algorithms, active trajectory states, track ages, Kalman filters, and appearance galleries are migrated rather than reset, preserving uninterrupted identity continuity.
When tested on held-out VisDrone2019 video sequences, Dynamic-SORT outperformed every static tracking configuration while maintaining superior operational efficiency. Relative to fixed BoTSORT, which achieved the highest marks among unswitched configurations with a Higher Order Tracking Accuracy of 47.8, a Multiple Object Tracking Accuracy of 68.9, and an ID F1-score of 78.9, Dynamic-SORT raised performance to 48.6 HOTA, 69.4 MOTA, and 80.1 IDF1. While these accuracy gains represent modest fractional improvements, the primary operational benefit manifested in system throughput: Dynamic-SORT delivered an end-to-end processing speed of 48 frames per second, compared to 40 frames per second for static BoTSORT. The amortized computational overhead required for calculating descriptors and evaluating rules amounted to just 1.8 milliseconds per frame, validating the efficiency of scene-conditioned execution.
The researchers explicitly cataloged several methodological constraints and limitations that define the current boundary of their findings. The sequential adaptation trials explored only a single transition step from VisDrone to Caltech, leaving multi-step catastrophic forgetting across extended task sequences uncharacterized. Furthermore, the framework was not benchmarked against dedicated continual learning baselines such as Elastic Weight Consolidation or experience replay, and the aerial-to-ground dataset shift does not substitute for validation on production autonomous driving suites like KITTI, Waymo Open Dataset, or nuScenes. Additionally, the baseline YOLOv8x detector recovers under half of annotated road users, leaving missed detections of small, occluded targets as the dominant failure mode. Future research will focus on closing the detector recall gap, testing continuous scoring functions to replace hard tracking thresholds, and evaluating closed-loop vehicular perception across diverse operational settings.
Subject of Research: Sequential adaptation and runtime multi-object tracking for vulnerable road user perception
Article Title: Sequential adaptation framework for efficient detection and adaptive tracking of vulnerable road users in real-world scenarios
Article References: Nasir, F. A., & Liaquat, S. (2026). Sequential adaptation framework for efficient detection and adaptive tracking of vulnerable road users in real-world scenarios. Discover Artificial Intelligence, 6(1), Article 1883. https://doi.org/10.1007/s44163-026-02414-6
Image Credits: AI Generated
DOI: 10.1007/s44163-026-02414-6
Keywords: Vulnerable Road Users, Sequential Adaptation, Dynamic-SORT, YOLOv8x, Object Detection, Multi-Object Tracking, Catastrophic Forgetting, Stochastic Gradient Descent, VisDrone Dataset, Caltech Pedestrian Dataset, Autonomous Driving Perception, BoTSORT, ByteTrack, DeepOCSORT, Higher Order Tracking Accuracy, BoxMOT, Real-Time Inference
Cite Scienmag News
Denise Maddox. (November 18, 2026). Adaptive Vision Framework Improves Road User Tracking While Reducing Model Forgetting. Scienmag. https://scienmag.com/adaptive-vision-framework-improves-road-user-tracking-while-reducing-model-forgetting/
Denise Maddox. "Adaptive Vision Framework Improves Road User Tracking While Reducing Model Forgetting." Scienmag, 18 November 2026, https://scienmag.com/adaptive-vision-framework-improves-road-user-tracking-while-reducing-model-forgetting/. Accessed 18 November 2026.
Denise Maddox. "Adaptive Vision Framework Improves Road User Tracking While Reducing Model Forgetting." Scienmag. November 18, 2026. https://scienmag.com/adaptive-vision-framework-improves-road-user-tracking-while-reducing-model-forgetting/








