The study, published in Frontiers in Built Environment, tackles a deceptively hard computer-vision problem. When an unmanned aerial vehicle photographs a construction site from a hundred meters up, a hard hat may occupy fewer than 20 by 20 pixels of a 4K frame. Lighting shifts from floodlit night scenes to harsh midday glare, weather degrades images with fog, dust, rain and snow, and helmet colors carry meaning: at many sites, color codes signal a worker’s role or compliance status. Conventional object detectors, including the widely used YOLO family, are trained on a fixed set of classes, so they stumble when a new helmet color appears or when sunset light makes a white helmet read as yellow. The research team, led by Abdul Razaque and Meenhoon Khan, argued that no single detector could handle all of these failure modes at once, and set out to prove that a carefully engineered combination could.

Their framework fuses two complementary detection paradigms. The first is YOLO-E, an open-vocabulary detector that aligns visual features with text embeddings from MobileCLIP, allowing it to recognize objects described by prompts such as “blue helmet” or “green helmet” without ever being trained on those specific categories. This zero-shot capability matters on real sites, where contractors may introduce new helmet colors to denote new job assignments. The second is YOLOv12, the newest closed-set member of the YOLO series, which uses an ELAN-T backbone with hierarchical window attention and a decoupled detection head to deliver high recall and roughly 25 percent lower inference latency than its predecessors. In the hybrid pipeline, YOLOv12 localizes helmets densely and quickly, while YOLO-E acts as a semantic verifier and generalizer that catches colors and equipment the closed-set model was never taught.

The most dramatic single component, however, is a technique called Slicing-Aided Hyper Inference, or SAHI. Instead of downscaling a high-resolution aerial image and destroying the tiny helmets within it, SAHI slices the frame into overlapping 640 by 640 pixel tiles with 20 percent overlap, runs the detector on each tile independently, then maps detections back to global coordinates and suppresses duplicates with non-maximum suppression. The authors provide a formal argument that this tiling guarantees recall for small objects can only improve: because each helmet occupies a larger effective scale inside a tile, its detection confidence rises, so fewer instances fall below the classification threshold. A tile-size sensitivity analysis confirmed the theory: performance climbed as tiles grew from 320 to 640 pixels, then dropped at 800 pixels, making 640 the sweet spot between accuracy and latency.

Before any detection happens, the pipeline conditions every frame through a three-stage normalization: automatic brightness alignment centers the HSV value channel’s mean at a stable midpoint, linear contrast stretching on the luminance channel expands the dynamic range without touching color information, and tone balancing clips the most extreme one percent of highlights and shadows. A lightweight mean filter suppresses sensor noise, which the authors prove reduces noise variance by a factor of the kernel size, and a synthetic weather module simulates fog, dust storms, rain streaks, snowfall and sun glare so the system can be stress-tested against domain shifts that would break naive detectors. The pipeline’s designers were explicit that they did not modify the internal architectures of either detector; all the gains come from preprocessing, tiling and fusion.

The results are striking. On a combination of a public personal-protective-equipment dataset and a newly collected drone dataset containing 6,471 annotated helmet instances captured at vertical, inclined and horizontal viewing angles, in day and night conditions and across five helmet colors, the full SAHI-enhanced hybrid framework lifted mean average precision at the 50 percent intersection-over-union threshold from 89 percent to 97 percent. Small-object recall, the metric that matters most for distant hard hats, nearly doubled from 0.41 to 0.80, and localization error fell to 2.25 pixels, far better than the 4.1 pixels of standalone YOLOv12 or the 7-plus pixels of two-stage detectors. Weather robustness and color fidelity scores reached 0.96 and 0.97 respectively, and the hybrid model outperformed Faster R-CNN, Mask R-CNN, RetinaNet and SSD across precision, recall and F1-score, with statistically significant improvements confirmed over multiple runs.

The ablation experiments reveal why each piece earns its place. Removing SAHI collapsed small-helmet recall to 0.41, as distant helmets vanished from downscaled frames. Removing color normalization dropped the color-fidelity score from 0.91 to 0.62 and made the system acutely sensitive to lighting shifts. Removing YOLO-E eliminated zero-shot recognition of unseen helmet colors entirely, while removing YOLOv12 caused recall to suffer in dense, cluttered scenes. The fusion mechanism itself, which weights the two detectors’ confidences with a tunable coefficient, peaked at a fusion weight of 0.6, where accuracy reached about 0.97, then declined as the balance tipped too far toward either model. Even weather testing carried insight: dust and fog proved the most damaging conditions for both detectors, while rain, snow and glare caused comparatively modest degradation.

The framework is not without costs. The full hybrid pipeline with SAHI runs at 12 to 20 frames per second on an NVIDIA RTX 4060, slower than the 41 to 74 frames per second of YOLOv12 alone, because each 4K frame becomes roughly nine tiles processed through two networks. The authors also acknowledge that their drone dataset came from a limited number of geographic locations, that red helmets proved hardest to detect because of their visual similarity to signs, toolboxes and worker clothing, and that the system has not yet been validated in a continuous real-world construction workflow with drone vibration and GPS drift. In severely occluded scenes or under dense fog, confidence in the closed-set branch fell even as its precision stayed high, and no fusion rule can recover an object that both detectors miss.

Still, the trajectory is clear. The team’s roadmap includes adaptive tile selection that slices only small-object regions, reinforcement-learning-based prompt optimization for the open-vocabulary branch, multiframe temporal reasoning to stabilize video streams, and multimodal fusion of RGB, thermal and infrared sensors for night operations. Because the open-vocabulary branch accepts arbitrary text prompts, the same framework can be extended to safety vests, goggles, gloves and harnesses without retraining. For an industry where a missed helmet can mean a death, the demonstration that open-vocabulary reasoning, high-recall closed-set detection, smart image slicing and disciplined preprocessing can be unified into a single real-time system marks a meaningful step toward autonomous, around-the-clock safety auditing on the world’s most dangerous workplaces.

Subject of Research: Hybrid open-vocabulary and closed-set deep learning for drone-based helmet detection and construction safety monitoring

Article Title: Hybrid open-vocabulary and closed-set helmet detection with slicing-aided hyper inference for robust drone-based construction-safety monitoring

Article References: Razaque, A., Bektemyssova, G., Khan, M., Alajlan, A. M., Keresh, A., Saltanat, N., Malika, Z., & Ayagoz, S. (2026). Hybrid open-vocabulary and closed-set helmet detection with slicing-aided hyper inference for robust drone-based construction-safety monitoring. Frontiers in Built Environment, 12, Article 1867168. https://doi.org/10.3389/fbuil.2026.1867168

Image Credits: AI Generated

DOI: 10.3389/fbuil.2026.1867168

Keywords: helmet detection, drone imagery, YOLO-E, YOLOv12, SAHI, construction safety, computer vision, open-vocabulary detection, personal protective equipment, image preprocessing, weather augmentation, UAV monitoring