Optimized DeepLabv3+ pipeline improves athlete boundary segmentation in four sports

An optimized DeepLabv3+ pipeline reported better pixel-level extraction of athletes’ outlines than the native DeepLabv3+ baseline in video frames from gymnastics, weightlifting, diving and sprinting. In the reported test results, the optimized model achieved a mean intersection over union (mIoU) of 88.31%, compared with 81.17% for the baseline, while boundary accuracy increased from 84.17% to 92.11%.

The computational study used sports footage drawn from the International Olympic Committee’s official historical image library, an open sports-action video collection on Kaggle and YouTube channels associated with major sporting events. The source pool contained 123,900 candidate frames, including transitional and static frames. After resampling and removal of unnecessary frames, a high-density motion sequence was assembled for model training and evaluation. The 78 action subtypes included vault, floor exercise, rings and parallel bars in gymnastics; snatch, clean and jerk and their stages in weightlifting; different jumping postures and entry angles in diving; and starting, acceleration and mid-run phases in sprinting.

Seven annotators with computer-vision backgrounds manually outlined athletes frame by frame in LabelMe 4.6.0, using polygonal contours and an average of 2.37 minutes per frame. No pretrained automatic segmentation or annotation-assistance tools were used. Three independent consistency tests on a randomly selected sample of 1.47 thousand frames produced Fleiss’ kappa values of 0.87, 0.89 and 0.88, with a mean of 0.88. The reference boundary was the visible outer contour of the athlete’s overall body, including the trunk, limbs and head; internal areas hidden by occlusion or equipment were not inferred.

For preprocessing, non-constant-frame-rate streams were resampled to 29.97 frames per second. Segments shorter than 0.5 seconds and static sequences with pixel variance below 12.7 across five consecutive frames were removed. Images were resized to 513 by 513 pixels with bicubic interpolation, and central cropping retained 80% of the frame. Channel-wise Z-score standardization was applied, followed by augmentation consisting of horizontal flipping with 50% probability, rotation of plus or minus 15 degrees, brightness and contrast adjustment from 0.85 to 1.15, and Gaussian noise with a standard deviation of 0.01.

The model used an Xception-65 backbone with ImageNet-pretrained weights. Its encoder combined depthwise separable and dilated convolutions, while the decoder fused high-resolution shallow features with deeper semantic features. The atrous spatial pyramid pooling configuration used dilation rates including 3, 7 and 13 to capture context at different scales. A fully connected conditional random field then refined the initial mask through five mean-field iterations, using pixel-position and RGB-color similarities. The network itself performed binary foreground-versus-background segmentation with one output channel; the 78 action labels were used to characterize and evaluate the dataset rather than included in the loss function.

Training used the AdamW optimizer, an initial learning rate of 0.0031, cosine-annealing decay to 1 × 10−6, a batch size of 12 and 83 epochs. Experiments ran in TensorFlow 2.13.0 with the Keras application programming interface on an NVIDIA Tesla V100 GPU with 32 GB of memory. The optimized pipeline’s reported mIoU was 7.14 percentage points above the native DeepLabv3+ baseline, equivalent to an 8.80% relative increase. Forward inference time decreased from approximately 2.03 to 1.42 milliseconds per frame across 17,470 test frames, corresponding to a reported 1.43-fold speedup. This timing comparison was separate from the segmentation-accuracy measurement.

Performance varied by action category. The sprint start produced the highest reported category-level results: 89.31% IoU, 94.17% boundary accuracy and 92.87% recall. The authors associated this performance with the athlete’s extended posture, clear limb contours and strong contrast against the background. Weightlifting clean and jerk reached 88.73% IoU, 93.41% accuracy and 91.19% recall. Gymnastics vault reached 84.19% IoU, while diving take-off reached 83.07% IoU, 88.13% accuracy and 86.43% recall. The study attributed the lower gymnastics result to rapid limb folding during aerial rotation and the lower diving result to gray similarity around splashes and edge truncation at water entry.

Comparisons with other architectures highlighted differences in measured speed and accuracy. U-Net had a reported inference delay of 14.31 milliseconds but obtained 72.03% IoU for diving. Mask R-CNN reached 87.13% segmentation accuracy for sprinting and had a 48.73-millisecond delay. The conditional random field’s Gibbs energy decreased during its five refinement iterations, reaching reported values of 0.43 × 10^4 for gymnastics, 0.31 × 10^4 for weightlifting and 0.19 × 10^4 for sprinting. These findings describe performance on the study’s frames and experimental setup rather than real-time judging in competition settings.

The study identifies limitations in the breadth and balance of its dataset. Samples were unevenly distributed among action categories, and coverage of partial occlusion, poor lighting and intense reflections remained limited. Although results across training, validation and test splits were reported as relatively stable, the evidence does not establish that any particular optimizer or learning-rate schedule caused that stability. The findings support further evaluation of this boundary-extraction approach as a visual input for action recognition, posture analysis, motion tracking and related sports-analysis tasks, but the paper does not demonstrate clinical benefits, systematic real-time deployment or superior performance in a broad range of unseen sports.

Subject of Research: Pixel-level athlete boundary extraction from gymnastics, weightlifting, diving and sprinting video frames

Article Title: Extraction of sports action boundaries using the DeepLabv3+ algorithm

Article References: Guo, R. (2026). Extraction of sports action boundaries using the DeepLabv3+ algorithm. Discover Artificial Intelligence, 6(1), Article 1245. https://doi.org/10.1007/s44163-026-02257-1

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02257-1

Keywords: DeepLabv3+, sports video analysis, athlete segmentation, boundary extraction, computer vision, gymnastics, weightlifting, diving, sprinting, conditional random field, Extraction, sports