Ceiling camera turns factory video into digital-twin state variables

A ceiling-mounted camera watching a real food-production floor has been turned into a working observation layer for a manufacturing digital twin, with every stage of the pipeline benchmarked on video the system had never seen during training. The study, published in Frontiers in Manufacturing Technology, formalizes a step that many camera-based industrial projects skip: how raw video becomes structured, time-stamped state variables that a digital twin can actually consume, rather than stopping at object detections or tracking boxes on a screen.

On the primary held-out test set of 150 manually reviewed frames containing 593 annotated boxes, the selected detector, YOLO11x fine-tuned with a freeze depth of 22, reached a mean average precision of 0.939 at an intersection-over-union threshold of 0.50 and 0.754 averaged across thresholds from 0.50 to 0.95. The stricter second metric penalizes weak localization, so the gap between the two numbers indicates that the model does not merely find objects but places boxes tightly around them. A secondary 60-frame subset drawn from the same hour with a different sampling phase confirmed the ranking, with YOLO11x reaching 0.936 and 0.821 on the two metrics respectively.

Model scale mattered even after domain adaptation. YOLO11x led on held-out data, while YOLO11l and YOLO11s emerged as lighter trade-off points, with YOLO11n the fastest at 2.2 milliseconds per frame but recall dropping to 0.780. The class-level picture stayed uneven throughout: vats were detected almost perfectly at 0.988 mAP50 on the held-out set, while people, appearing under more varied poses, occlusions and machine interactions, reached only 0.891, making person detection the limiting part of the perception layer.

A transfer-depth ablation delivered one of the study’s most instructive findings. Freeze depth 10 produced the best validation score, 0.954 mAP50, but fell to 0.894 on the unseen hour, and full unfreezing degraded further to 0.836. The shallowest adaptation, freezing 22 layers, preserved the strongest held-out performance. The explanation is practical: the dataset is small and strongly biased toward one ceiling-view geometry, so deeper retraining fits the annotation pool tightly while eroding the general visual priors inherited from large-scale pretraining. For industrial teams with limited labeled data and sharply off-distribution viewpoints, constrained transfer beat deeper adaptation on the data that actually mattered.

ByteTrack reached a HOTA score of 0.783, MOTA of 0.718, IDF1 of 0.787, 15 identity switches and 27.6 frames per second, while BoT-SORT managed 0.767, 0.719, 0.786, 17 switches and 17.0 frames per second. With a fixed overhead camera and no cross-camera re-identification required, the lighter tracker won on both association quality and runtime, and the denser, more interaction-heavy clip concentrated nearly all observed identity switches.

The full-horizon run applied the selected YOLO11x and ByteTrack stack to the entire held-out 60-minute recording and produced zone-level analytics that read like an operational fingerprint of the floor. The machine service area dominated with 276 entries, 172 exits, an average occupancy of 1.36, an occupied-time ratio of 95.2 percent and 4,911.8 seconds of total dwell time. The front vat staging area ranked second with 204 entries, 112 exits, 68.9 percent occupied time and 4,001.3 seconds of dwell. The rack buffer area, right loading area and left transit lane accumulated 1,572.0, 1,029.0 and 550.0 seconds of dwell respectively, cleanly separating two dominant zones from three peripheral ones.

The long run also exposed an honest limitation. The tracker generated 639 unique track identities over the hour, far more than the number of physical entities on the floor, reflecting identity fragmentation rather than true object counts. Aggregate occupancy and dwell summaries remain usable at this timescale, but event counts that depend on persistent identity cannot be treated as exact, and re-identification or trajectory stitching will be needed for identity-stable long-horizon analytics. Other constraints include a single annotator, so inter-annotator agreement was not estimated, one camera and one facility, a bounding-box-center zone-membership rule that suits vats better than workers, and unavailable camera intrinsics that limit exact geometric replication at new sites.

The authors are careful about what they have and have not built. The validated contribution ends at the interface that exports occupancy, entries, exits, dwell time and activity concentration; simulation, prediction, data fusion with other sensors, bidirectional synchronization and closed-loop feedback remain downstream work, and the implementation demonstrates observation on recorded video rather than a continuously deployed real-time monitoring service. Within those boundaries, the study offers other researchers a reproducible path: a layered mapping from production video to twin-ready state variables, an evidence-based rule for choosing detector scale and transfer depth under limited data, and a demonstration that a single overhead camera can yield meaningful operational intelligence about where work concentrates on a factory floor.

Subject of Research: A vision-based observation layer that converts ceiling-mounted production video into digital-twin state variables, benchmarked on held-out food-factory footage.

Article Title: A vision-based observation layer framework for manufacturing digital twins: detection, tracking, and zone analytics from ceiling-mounted production video

Article References: Amirkhanova, G., Amirkhanov, A., Amirkhanov, B., Faruzkyzy, Y., Fu, Y., & Zhasuzakova, M. (2026). A vision-based observation layer framework for manufacturing digital twins: detection, tracking, and zone analytics from ceiling-mounted production video. Frontiers in Manufacturing Technology, 6, Article 1875591. https://doi.org/10.3389/fmtec.2026.1875591

Image Credits: AI Generated

DOI: 10.3389/fmtec.2026.1875591

Keywords: digital twin, manufacturing, computer vision, object detection, multi-object tracking, YOLO11, ByteTrack, zone analytics, food production, ceiling-mounted camera, observation layer, held-out evaluation