YOLOv5-LiDAR basketball robot reaches 97.85% detection accuracy in evaluation

A basketball-robot perception and control system combining YOLOv5 image detection with LiDAR spatial sensing reached 97.85% detection accuracy in evaluations involving 500 basketball video sequences, according to the study. The system, which was also tested in simulation-based interception and cooperative-defense tasks, recorded a 6.12% tracking-loss rate, a 94.28% autonomous-interception success rate and a 92.56% cooperative-defense success rate. These results came from a technical evaluation rather than a clinical or field deployment study, and the researchers noted that performance under extreme motion blur and dense occlusion still requires further investigation.

The researchers developed the system in stages. Its initial YLM algorithm combined YOLOv5, which identifies basketballs, players and players’ hands in camera images, with LiDAR, which supplies three-dimensional positional information. A confidence-driven fusion mechanism adjusted the contribution of each sensor according to visual detection confidence and the completeness of the point cloud. Multi-object tracking and trajectory prediction were then used to connect detections across frames and estimate where moving targets were likely to go, helping the system maintain target identities through temporary interruptions. The evaluation compared this composite system with YOLOv5 plus a conventional Kalman filter, LiDAR plus a support vector machine, and a convolutional neural network plus optical flow.

Several specialized components were added to improve performance in crowded, fast-moving basketball scenes. The visual detector included a high-resolution branch intended to catch small basketballs and hands, along with a lightweight convolutional block attention module. A robust Kalman filter based on kernel-density estimation replaced the conventional filter to reduce trajectory divergence caused by measurement outliers. The complete DRS-YLM model also incorporated camera–LiDAR calibration and timestamp synchronization, cross-modal feature alignment, graph-based modeling of interactions among players and the ball, and a hierarchical proximal policy optimization control strategy. These additions extended the initial YLM architecture from perception and tracking into a pipeline designed to produce robot control commands.

Sensor synchronization was central to the model because the camera and LiDAR did not simply contribute equivalent information at every moment. YOLOv5 produced image-space bounding boxes and confidence scores, while LiDAR point clouds were clustered to estimate spatial positions. The system mapped image detections into the LiDAR coordinate system and combined the two streams into a target state. A contrastive alignment loss mapped visual and point-cloud features into a shared semantic space, while the final control policy combined a reinforcement-learning policy with interaction-adaptive policies weighted by relationships among targets. The overall training objective combined detection, tracking and trajectory-prediction losses with balancing coefficients of 0.5, 0.3 and 0.2, respectively.

Testing used an open basketball-game video dataset for training and multiscene basketball-robot measurement data for validation. The source material states that both captured conditions resembling the physical robot’s camera and LiDAR hardware, including indoor and outdoor settings, changes in illumination, crowded scenes and occlusions. Model training used 300 epochs, Adam optimization, an initial learning rate of 0.001 and a batch size of 16. The camera operated at 1280×720 resolution and 30 frames per second, while the LiDAR had an effective range of 0.1–15 metres and sampled at 10 Hz. The validation procedure also incorporated sensor sampling delay and communication loss to reflect conditions encountered during data processing, and calibration parameters were adjusted with reference to the physical equipment’s installation structure.

Across the 500-sequence evaluation, the YLM algorithm initially recorded an 85.67% detection rate and a 9.34% tracking-loss rate. Its detection rate subsequently rose to 97.85% and stabilized at the 313th processed segment, while tracking loss fell to 6.12% by the 124th segment. The final detection rate of the YOLOv5-KF comparison was 92.45%, while the LiDAR-SVM comparison had a 15.78% tracking-loss rate. In a repeated comparison involving 10 independent runs and 100 randomly selected sequences per run, YLM produced a target detection precision of 96.72% ± 0.48%, a tracking-loss rate of 6.85% ± 0.31% and a task success rate of 93.64% ± 0.47%. One-way analysis of variance and Tukey HSD comparisons indicated statistically significant differences among the methods at the stated 0.05 significance level.

Tracking tests covered fast dribbling, multiperson occlusion, long-range three-point shots and scrambles beneath the basket. YLM’s tracking success rates were 97.52%, 97.78%, 97.13% and 97.89%, respectively, while its trajectory-consistency scores ranged from 96.38% to 98.62%. In a separate target-recognition test, it achieved peak localization accuracy of 97.79%, compared with 85.24% for YOLOv5-KF and lower values for the other two comparisons. During autonomous-interception simulation, YLM achieved 95.67% target recognition, a mean trajectory prediction error of 0.12 metres and a 94.28% task success rate. In cooperative defense, it achieved 93.78% recognition, a 0.15-metre prediction error, 48.7 milliseconds of end-to-end response delay and a 92.56% task success rate.

The expanded DRS-YLM model was also evaluated on tracking and decision tasks. On training data, its task completion rates were 98.45% for single-person tracking, 97.28% for distinguishing multiple people, 98.34% for shot prediction and 98.91% for interception decision-making. On validation data, decision accuracy reached 95.67% for shot prediction and 94.12% for interception decisions. Its predicted shot trajectory was reported to have 98.74% accuracy, compared with 88.21% for YOLOv5-KF, 84.39% for LiDAR-SVM and 92.68% for CNN-OF. Mean training loss for target tracking was 0.074, while mean validation loss for decision-making was 0.089; both were lower than the corresponding comparison values, with the study reporting p-values below 0.001.

Scenario tests assessed the complete model under fast offense and defense, multiperson confrontation and complex occlusion. In fast offense-and-defense situations, DRS-YLM reached 98.76% target detection precision, 97.58% target recognition, 96.87% interaction-modeling accuracy and a system response efficiency of 28.4 frames per second. Under multiperson confrontation, the corresponding results were 97.94%, 96.83%, 95.62% and 26.8 frames per second. With complex occlusion, it recorded 97.52% detection precision, 96.41% recognition, 95.18% interaction-modeling accuracy and 26.3 frames per second. These measurements showed that the complete system maintained high reported accuracy across the tested conditions while producing different response-efficiency values in each scenario.

The study’s principal claim is that complementary visual, spatial, temporal and interaction information can make a basketball robot’s perception and decisions more reliable than the tested single-modal or simpler comparison architectures. The final model combined calibrated YOLOv5-LiDAR perception, cross-modal feature alignment, multi-object tracking, trajectory prediction, graph-based interaction analysis and reinforcement-learning control in one pipeline. However, the evidence was based on public or measured datasets, controlled comparisons and simulation-based robot tasks using specified hardware and parameters. The authors also identified severe motion blur and densely overlapping interactions as unresolved conditions, and called for future work on multi-agent modeling and cross-modal self-supervised learning. The results therefore demonstrate promising performance under the study’s evaluated scenarios, while broader generalization to unpredictable real-world basketball games remains to be established.

Subject of Research: Evaluation of YOLOv5-LiDAR fusion, trajectory prediction and reinforcement-learning control for basketball robots

Article Title: Perception and decision optimization of basketball robot based on YOLOv5 and LiDAR fusion

Article References: Li, R. (2026). Perception and decision optimization of basketball robot based on YOLOv5 and LiDAR fusion. Discover Artificial Intelligence, 6(1), Article 1178. https://doi.org/10.1007/s44163-026-02133-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02133-y

Keywords: basketball robots, YOLOv5, LiDAR, sensor fusion, multi-object tracking, trajectory prediction, reinforcement learning, robot control, sports robotics, cross-modal alignment, Perception, decision