Seeing Ripeness Through Smart Glasses: Lightweight AI Brings Real-Time Harvest Decisions to Cherry Tomato Fields

In greenhouses where cherry tomatoes hang in dense, multi-colored clusters, the difference between a perfectly timed harvest and a costly one can come down to subtle shifts in skin color that human eyes, fatigued after hours of picking, routinely misjudge. A research team led by Yang Shao, Jiale Cui, Chen Tang, Zhenqing Liu, Fengjiao Xu, Ruifang Zhai, Wanneng Yang, and Peng Song has now built a system that takes that judgment away from tired eyes and hands it to a pair of augmented reality smart glasses. Published in the journal Artificial Intelligence in Agriculture, their work describes a complete, wearable pipeline that looks at a tomato truss, decides how ripe it is, grades it, and paints the verdict directly into the wearer’s field of view — all in real time, without a phone, a server, or a robot.

The core challenge the team set out to solve is deceptively specific but commercially significant. While most existing computer-vision systems for tomato maturity focus on large, individually harvested fruits, cherry tomatoes destined for supermarkets are typically picked and sold as entire trusses. That means a harvester needs to evaluate two things simultaneously: the maturity of each tiny fruit on the stem, and the overall ripeness profile of the cluster as a single harvesting unit. Conventional cascaded approaches — one model to find the truss, another to classify individual fruits, a third to aggregate the results — simply burn too much power and introduce too much latency for a battery-constrained wearable device. The researchers’ answer was to collapse everything into a single, lightweight detection network they call DGTruss-YOLO, built on an improved YOLOv11 architecture.

DGTruss-YOLO incorporates three interlocking technical innovations. The first is a Spatial and Channel Reconstruction Convolution, or ScConv, embedded in a redesigned module that replaces the baseline C3k2 structure in YOLOv11’s backbone. ScConv works on a “reconstruct-before-convolution” principle: a Spatial Reconstruction Unit uses Group Normalization statistics to split feature maps into information-dense and information-sparse regions and recombines them, while a Channel Reconstruction Unit employs a split-transform-fuse strategy with group-wise and point-wise convolutions to strip out redundant channel responses. The result is a network that encodes less waste and more signal per unit of computation — a crucial property when every millijoule counts.

The second innovation tackles the multi-scale problem. A single cherry tomato and an entire truss differ enormously in geometric size, and standard Feature Pyramid Networks often handle such disparities poorly, suffering from shallow-layer semantic poverty and redundant cross-scale fusion. The team’s MFM-MAFPN module addresses this with two auxiliary fusion paths: a Shallow-layer Supplementary Auxiliary Fusion module that injects high-resolution detail into deeper semantic pathways, and a Deep-layer Supplementary Auxiliary Fusion module that aggregates features across parallel branches with an equal-proportion channel allocation strategy that preserves guidance information from earlier layers. Between fusion nodes, a Multi-scale Feature Modulation block replaces static concatenation, using global average pooling and a small perceptron to adaptively reweight channels so the network can emphasize whichever scale carries the most decision-relevant information at any moment.

The third piece is a deployment-oriented detection head called Efficient_Detect. Rather than simply swapping depthwise separable convolutions for standard ones, the researchers streamlined the entire head topology into a compact, decoupled two-branch design — one branch for bounding-box regression, one for classification — built from standard convolution blocks that fuse cleanly with BatchNorm and SiLU activations. The reasoning here is pragmatic: on mobile inference backends like NCNN, which runs on the AR glasses’ Qualcomm Snapdragon chip, regular convolutions support efficient operator fusion and low-precision FP16 or INT8 optimization far more reliably than fragmented, hardware-sensitive operators. The team trained the model with a combined loss comprising CIoU-based box regression, binary cross-entropy classification, and distribution focal loss, keeping the training objective consistent with the YOLOv11 framework.

The numbers validate the lightweighting strategy. Compared with the baseline YOLOv11, DGTruss-YOLO reduces parameters by 11.07 percent, computational load by 12.70 percent, and model size by 9.62 percent, while actually maintaining a mean average precision at an IoU threshold of 0.5 of 0.8539. That combination — smaller, faster, and no less accurate — is the kind of trade-off that makes or breaks edge deployments in agriculture, where hardware budgets are thin and greenhouse conditions are hostile to computation.

Training data came from the Qingdao Kaisheng Haofeng Smart Agriculture Base, where the team collected 1,200 raw images with an Azure Kinect depth camera at 1920 by 1080 resolution, deliberately sampling across morning, noon, and late-afternoon light to capture front-lighting, backlighting, strong reflections, leaf occlusion, and cluttered backgrounds. Images were captured at a consistent camera-to-target distance of roughly 40 centimeters. After careful preprocessing — resizing to 512 by 512 pixels with gray padding, and an automated OpenCV-based annotation cleanup that cut the label error rate from 3.2 percent to 0.5 percent — synchronized augmentation with geometric and illumination transformations expanded the dataset to 2,864 images split into 2,281 training and 583 validation samples.

At the fruit level, the system classifies individual tomatoes into three stages — Immature, Turning, and Ripe — based on skin color evolution and pedicel morphology. The genuinely novel step happens afterward: a weighted fusion formula converts those fruit-level counts into a single truss-level maturity index. Ripe fruits count as 1.0, Turning fruits as 0.5, and Immature fruits as zero, so the index I equals the number of ripe fruits plus half the number of turning fruits, divided by the total fruits on the truss. That continuous index, ranging from zero to one, is then mapped onto five practical harvest grades, A through E, reflecting real-world cold-chain and logistics requirements. Grade A trusses, dominated by ripe fruit, can go straight to market; lower grades signal that waiting is the better business decision.

Deployed on commercially available SUPERHEXA AR smart glasses running Android, with the model converted through ONNX and optimized via the pnnx tool into NCNN-compatible parameter and binary files, the system achieves stable real-time inference at 15 to 20 frames per second. On real greenhouse evaluation data held separate from training and validation, the wearable system reached a truss-level maturity assessment accuracy of 95.26 percent, with 100 percent adjacent-tolerance accuracy — meaning that even when it erred, it never jumped more than one grade. Inter-rater statistics reinforce the reliability: a Cohen’s Kappa coefficient of 0.9399 and a weighted F1-score of 0.9528.

The broader significance lies in the human-machine collaboration paradigm the work champions. Fully autonomous robotic harvesting remains expensive, fragile, and years away from scale, particularly for delicate crops like cherry tomatoes. By contrast, a pair of smart glasses costs a fraction of a harvesting robot, leaves the dexterity and agronomic judgment of human pickers fully intact, and simply augments their perception. A novice worker wearing the glasses sees color-coded boundaries around each truss and an on-screen recommendation — pick now or wait — effectively compressing years of harvesting experience into a wearable overlay. The system forms a closed perception-decision-feedback loop entirely on the device, with no reliance on external terminals or cloud connectivity, an important consideration in rural infrastructure contexts.

There are still mountains to climb before such systems are ubiquitous in commercial greenhouses. The current evaluation covers one crop, one cultivar class, and one hardware platform; occlusion from dense canopies, motion blur from a moving wearer’s head, and seasonal variation in fruit morphology will all stress the model further. Battery life under continuous inference, user ergonomics over full shifts, and integration with farm management software remain open engineering questions. But the study demonstrates something that earlier efforts only hinted at: that a carefully engineered, hardware-aware deep learning model can run the entire perception-to-decision pipeline for a genuinely dual-scale agricultural task on a consumer wearable, at interactive frame rates, with accuracy high enough to trust. As the authors frame it, this is not a replacement for human harvesters but a bridge — a practical, low-threshold upgrade path that moves labor-intensive horticulture from experience-driven guessing toward data-driven precision, one pair of glasses and one tomato truss at a time. For an industry under pressure from labor shortages and rising quality demands, that bridge may arrive exactly when it is needed most.