The technical heart of the system is a customised deep learning model the team named RGD-YOLO. It builds on the YOLOv10m object detection architecture, which combines feature pyramids and multi-scale training to spot small objects, and adds two purpose-designed modules. The first, a C2f_FADC block based on frequency-adaptive dilated convolution, adaptively widens the model’s receptive field so that it preserves the high-frequency signals that mark grain edges and texture while suppressing low-frequency background noise from stems and reflections. The second, an SPPF-LSKA block, applies large-kernel depthwise convolutions with spatial attention, allowing the network to lock onto individual grains even when spikelets are densely packed and partially hidden. In attention heatmaps, the baseline YOLOv10m model tended to light up grain edges and panicle tips when grains overlapped, effectively guessing from peripheral cues, whereas RGD-YOLO continued to focus squarely on the grains themselves. In ablation tests, these modifications pushed the model’s strict COCO-benchmark detection score (mAP@0.5–0.95) to 82.2 percent, with precision of 98.97 percent and recall of 98.54 percent, while keeping the model a compact 34.6 megabytes — small enough for practical cloud deployment.

Detection alone is not enough, however, because meaningful measurements of grain length, width, perimeter, roundness, and seed-coat colour can only be taken from grains whose full outlines are visible. To solve this, the researchers paired a fine-tuned EfficientNet classifier with the Felzenszwalb superpixel segmentation algorithm. Each grain detected by RGD-YOLO is converted into a pseudo-coloured image using Otsu thresholding and JET colour mapping, segmented at a systematically optimised ratio, and then judged by the classifier as either a complete “whole grain” or an occluded fragment. Out of 885,011 grains detected across the study’s image collection, the pipeline confidently identified 77,812 whole grains. Those grains, together with their occluded counterparts, form the backbone of a new publicly released training resource called the World Rice Grain Dataset, or WRGD, which began with 50,000 whole grains and 50,000 occluded grains after augmentation and now contains 77,812 verified whole-grain images spanning 864 landraces.

The imaging workflow underlying the platform is deliberately frugal. Field workers place a rice panicle on a black matte board alongside a red reference label and a QR code, then photograph it from about half a metre above with an ordinary mid-range Android phone — Xiaomi Mi 11, Huawei Nova 7 SE, and vivo Y50 handsets were all used across the study, demonstrating that sensor differences between manufacturers do not break the analysis. The QR code carries accession metadata, the red label calibrates colour and provides a pixel-to-millimetre conversion factor, and histogram matching against a benchmark image harmonises exposure across devices. Roughly 64 gigabytes of imagery — 10,837 photographs covering 322 landraces in the 2020 season and 709 in 2022, plus 176 recombinant inbred lines — were collected at a throughput of about fifteen seconds per panicle, with no specialised camera, scanner, or lighting rig required.

The platform itself is hosted openly at rgpas.cemps.ac.cn and built on the Django framework following a Model-View-Controller architecture, with a logic layer for workflow control, an AI model layer housing the trained detectors, a service layer for visualisation and storage, and a MySQL-backed digital grain library. Users can run single-image analysis for quick grain counting, batch-process entire image series for panicle- and grain-level statistics, and browse a searchable database containing genotypic and phenotypic records for 864 landraces and 176 recombinant inbred lines, drawn from 1092 panicle images and 108,547 whole grain images. Registration is restricted to academic email addresses, results export to CSV, and the interface works in both English and Chinese. The source code and trained models are maintained on GitHub, and the WRGD dataset is archived at the EBI BioImage Archive, a level of openness the authors argue is essential for reproducibility.

To prove the platform’s numbers can be trusted, the team validated RGPAS-derived traits against ground truth in two ways: manual counting of grains per panicle on 1012 lines by field specialists, and benchmark measurements from a commercial SC-E rice appearance quality analyzer for grain size, shape, and colour traits. The agreement was strong, and the phenotypic distributions the platform produced matched well-established biology — indica varieties showed roughly 10 percent greater grain perimeter and longer, more slender grains, while japonica grains were shorter and wider, with overall grain size remaining statistically similar between the subspecies. When applied to the 176 recombinant inbred lines, a population the detection model had never seen during training, the platform again produced clean, near-Gaussian trait distributions, indicating that its generalisation was not an artefact of overfitting to the landrace panel.

The most consequential test, though, was whether photographs processed in the cloud could feed genuine genetics. For genome-wide association studies, the team drew on high-depth whole-genome resequencing data at roughly 14-fold coverage, filtering variants to retain 3,583,878 high-confidence biallelic SNPs and running the EMMAx mixed-model pipeline on 322 landraces from 2020 and 709 from 2022. The results were striking. RGPAS-derived grain length-to-width ratio pointed squarely at GS3 on chromosome 3 and GW5 on chromosome 5, the two canonical regulators of rice grain shape, with association signals reaching −log10P values above 26 in the 2022 data — and the same loci appeared when the traits were measured conventionally after harvest. Grain length analyses recovered GS3, OsSPL13, TGW6, and the flowering regulator Hd1; grain width recovered GW5, GW7, GE, and GW2; and grain number per panicle associated with OsCKX2, GS5, GRF4, OsMADS1, TGW6, and GATA28, among others. Even seed-coat colour, quantified from the images’ green channel, mapped to the Rc and Rd genes that govern pericarp pigmentation and proanthocyanidin biosynthesis in red- and black-grained rice.

Quantitative trait locus mapping in the recombinant inbred lines, built on a genetic map of 1,148 recombination bins spanning 2,499 centimorgans, reinforced the picture. RGPAS-derived traits detected QTLs containing D61, Sd1, GS3, GW5, OsMADS50, and OsMADS56, and — in a notable detail — the computational grain number trait recovered a locus near Hd1 on chromosome 6 that the manually counted trait failed to detect, while manual scoring alone picked up GAD1 on chromosome 8. Both approaches independently flagged an unknown locus on chromosome 11 explaining about 10 percent of phenotypic variation, and a second unknown locus on chromosome 6 explaining 9.3 percent of grain width variation, both flagged as promising candidates for future fine-mapping. The researchers note that loci consistently detected across both the 2020 and 2022 seasons, and across both the landrace and RIL populations, were treated as high-confidence associations, with quantile-quantile plot inspection confirming that genomic inflation reflected a handful of strong-effect loci rather than systematic confounding.

The team is candid about the platform’s limits. Cloud operation requires a stable internet connection, performance can suffer under concurrent load or limited bandwidth, and traits such as seed-setting rate, plump-to-shrivelled grain ratios, and spatial grain-filling patterns remain beyond its current scope. Looking forward, the authors envision edge-cloud hybrid computing that would keep AI analysis running smoothly in fields with patchy connectivity, integration of environmental data into an interconnected genotype–environment–phenotype framework, extension to panicle architecture traits such as branch number and grain distribution, and — thanks to the system’s modular design — straightforward adaptation to other cereals, including wheat and maize. For now, though, the significance of the work is hard to overstate. By chaining together a cheap smartphone, a lean and purpose-built neural network, and free cloud infrastructure, RGPAS lowers the cost of high-quality phenotyping to near zero and hands plant breeders a direct pipeline from a photo in the field to a candidate gene on the genome — precisely the kind of tool that grain-focused crop improvement, under the pressures of a changing climate, has been waiting for.

The core difficulty the researchers confronted is threefold. Training data in agriculture is limited because photographing and annotating crops is slow and expensive, often requiring human labor in polytunnels and orchards. Fruit appearance varies dramatically across growth stages, so a tomato early in ripening looks little like the mature fruit a harvester ultimately needs to find. And occlusion is pervasive, since leaves, stems and neighboring fruit routinely hide much of the visual information a detector would like to see. Conventional detectors, whether anchor-based architectures such as Faster R-CNN and SSD or anchor-free approaches that predict object centers per pixel, typically assume fixed anchor configurations tuned on generic benchmarks and demand substantial labeled data. In unstructured agricultural scenes, those assumptions break down precisely where they are needed most.

At the heart of the new framework is the Fruit Discriminative Network, or FDN, a clever piece of weakly supervised engineering that sidesteps the need for exhaustive bounding-box annotation. The FDN is built on a VGG-16 backbone pretrained on ImageNet and performs only a simple binary classification: does an image contain fruit or not? Training images are drawn from two pools, a Without Fruit set of plants, leaves and backgrounds gathered from web searches, and a Detected Fruit set containing strawberries, tomatoes or a mixed collection of eleven fruits including apple, apricot, clementine, kaki, lemon, mango, orange, peach, pear, plum and strawberry. Because the task is binary, the network needs no pixel-level labels, and twenty epochs of training suffice.

The technical magic happens after classification, using Gradient-weighted Class Activation Mapping, or Grad-CAM. This technique computes the gradients flowing back to the final convolutional feature maps, weights the forward activations accordingly, and produces a spatial importance map showing which pixels most influenced the fruit-versus-no-fruit decision. Mathematically, the attention at each grid position is the sum over feature channels of the class-specific feedback gradients multiplied by the last-layer activations, passed through a rectified linear unit so that only positive contributions remain. The resulting heatmap, rendered across separate RGB channels, reveals exactly where the model believes fruit to be. Two kinds of Euclidean distance measurements are then extracted from this map: the distance between prominent highlighted regions and their non-important surroundings, and the distance between adjacent highlight areas. These distances provide a rough estimate of fruit size, which in turn encodes fruit type and growth state without any manual measurement.

Those size estimates feed directly into the second innovation, adaptive anchor box generation. In the standard SSD detector, six so-called anchor maps are defined by fixed lower and upper scale limits derived from hand-tuned hyperparameters, with a minimum scale computed from a base dimension of 300 pixels and ratios drawn from a predetermined set. The Lincoln team observed that such uniform settings are rarely optimal when fruit size distributions differ so widely between crops. Their algorithm inspects the attention map, identifies salient and non-salient pixel positions, and measures the Euclidean distances among them. If fruits tend to be gathered closely, as with clusters of grapes or densely packed tomatoes, distances within the salient set best reflect individual fruit size; if fruits are scattered, distances between salient and non-salient points are more informative. Based on this density assessment, the system selects one of two strategies. A frequency-based method suits datasets where most objects fall in a narrow size range: it takes the six most frequently occurring distances as the minimum scales of the six anchor maps and doubles them for the maximum scales. A linear-based method handles datasets with wide size variation, spacing the six anchor scales evenly between the minimum and maximum observed distances. Ablation experiments confirmed the intuition, with the frequency method performing better on the relatively uniform strawberry dataset and the linear method winning on tomato and multi-fruit data.

The third pillar is transfer learning, and here the study makes a contribution that goes beyond the usual practice by systematically exploring how the sizes of both source and target training sets affect performance. The team used the tomato dataset, collected from a garden and showing multiple growth stages, as the source domain, then fine-tuned on strawberry images from a commercial polytunnel in Lincoln, UK, and on the multi-fruit web collection. Fine-tuning froze the VGG convolutional layers and the localization layers while unfreezing the confidence layers, preserving initial weights there before continued training. Tomato training images were partitioned into subsets of 49, 99, 198 and 396 images, strawberry into subsets from 40 to 198 images, and multi-fruit into subsets from 40 to 336 images, each evaluated against fixed test sets of 50 strawberry and 111 multi-fruit images.

The results tell a nuanced story. Transferring knowledge from tomatoes to the multi-fruit dataset yielded the strongest gains, with the best combination of 336 target images and 396 source images reaching a mean Average Precision of 0.6409, nearly a 10 percent improvement over the non-transfer baseline. For a fixed tomato model trained on 198 images, raising multi-fruit training data from 40 to 120 images boosted mAP by 0.21, whereas the equivalent strawberry increase added only 0.03. To explain the asymmetry, the researchers visualized feature spaces extracted by VGG-16 using t-SNE, which converts similarities between data points into joint probabilities and minimizes the Kullback-Leibler divergence between high-dimensional data and a low-dimensional embedding. The two-dimensional projections showed that the tomato feature distribution largely covers that of the multi-fruit set, while strawberries, photographed against complex and cluttered backgrounds, occupy a much wider feature space, making them harder to transfer to.

Qualitatively, the detector’s outputs also proved more useful for actual farm robotics. Where the standard SSD often captured only part of a strawberry, offering insufficient information for a gripper, ADNet’s prediction boxes frequently included the stem, which is exactly what a picking mechanism needs to grasp. The adaptive anchors even located fruits whose color nearly matched the background, and handled clustered fruits markedly better. Interestingly, the Grad-CAM analysis uncovered a quirk: for tomatoes, the most discriminative regions were the stems rather than the fruit itself, because stems reliably co-occur with dense tomato clusters yet rarely appear in the fruitless background images. Rather than treating this as a failure, the researchers exploited it, using within-highlight distances to estimate the spacing between clustered tomatoes.

Efficiency figures round out the picture. ADNet needed only 3,000 training iterations to match the accuracy SSD achieved after 5,000 on the strawberry and tomato datasets, and loss curves fitted with twentieth-degree polynomials showed smoother, faster convergence on tomato and multi-fruit data. The FDN stage does add overhead, roughly 600 seconds for strawberries, 1,300 seconds for tomatoes and 235 seconds for the multi-fruit set, but the authors argue this one-time cost is offset by faster detection convergence and reduced annotation demands.

The work, supported by the EPSRC Centre for Doctoral Training in Agri-Food Robotics, positions ADNet_TL as a practical foundation for fruit forecasting and selective harvesting under genuinely unstructured conditions. The authors are candid about limitations, noting sensitivity to hyperparameter choices and residual domain-shift effects between source and target datasets. Their planned next steps include self-tuning hyperparameter techniques and adversarial and unsupervised domain adaptation, which could allow a single detector pretrained on one crop to generalize to many others with almost no local annotation at all.

Traditional methods for rice panicle counting have typically employed detection-based, density-based, or location-based strategies, each with inherent shortcomings. Detection models often fail to perform adequately in crowded or occluded scenes due to their reliance on individual object identification. Density-based approaches, which convert images into spatial density maps, are prone to degradation caused by complex backgrounds and environmental noise. Location-based models like P2PNet attempt to directly pinpoint panicle centers, offering interpretability and computational simplicity; however, their limited receptive fields and sensitivity to label inaccuracies constrain their robustness in heterogeneous field conditions.

LKNet represents a pivotal evolution from these earlier models by integrating large-kernel convolutional blocks—referred to as LKconv modules—that profoundly expand the network’s receptive field. This architectural innovation allows LKNet to dynamically adjust its perceptual scope according to the spatial scale of rice panicles, which vary considerably with panicle types and phenological stages. Furthermore, the incorporation of a customized localization loss function enhances the model’s tolerance to annotation errors and structural variability, markedly improving counting precision in complex, real-world scenarios.

The comprehensive evaluation of LKNet involved extensive comparative benchmarking across diverse datasets, covering both crowd counting and agricultural domains. Notably, on the challenging ShanghaiTech PartA crowd dataset—characterized by dense and cluttered distributions—LKNet achieved a mean absolute error (MAE) of 48.6 and root mean square error (RMSE) of 77.9, surpassing the original P2PNet and the detection-based PSDNN_CHat framework. On the less dense PartB dataset, LKNet matched state-of-the-art results, showcasing its versatility across varying crowd densities.

Transitioning to crop-specific tasks, LKNet demonstrated exceptional performance in rice panicle counting, exhibiting an RMSE of 1.76 and an R² coefficient of 0.965. This level of accuracy outperformed rival models that excelled in counting larger targets such as maize tassels, underscoring LKNet’s specialized benefits for fine-grained agricultural phenotyping. When applied to UAV-acquired rice canopy images captured at an altitude of seven meters, the model consistently maintained R² values above 0.98 across diverse panicle morphologies—compact, intermediate, and open—evidencing its adaptability to spatial and phenotypic diversity in field conditions.

An observed limitation emerged during later growth stages when increased occlusion and morphological variation introduced some decline in counting accuracy. This phenomenon highlights the inherent complexity of natural crop environments and further emphasizes the necessity for models like LKNet that can dynamically recalibrate their receptive scope and loss parameters in response to evolving scene characteristics.

Ablation studies delved deeper into the architectural contributions of the LKconv backbone, revealing its significant role in elevating both accuracy and computational efficiency. Integration of this backbone reduced RMSE dramatically from 2.821 to 0.846, while also halving the number of model parameters. Among various large-kernel configurations tested, the sequential large-kernel module equipped with an attention mechanism exhibited the highest correlation with ground truth labels, boasting an R² of 0.993. This fusion of attention and large kernels enables the model to capture subtle interrelations and suppress extraneous background features effectively.

Beyond numerical metrics, interpretability analyses via class activation mapping unveiled LKNet’s enhanced localization capabilities. Compared to P2PNet, LKNet exhibited broader and more contiguous focus areas around panicle centers, demonstrating superior background suppression and target discrimination even in visually complex scenes. These qualities are paramount in agricultural contexts, where precise localization informs downstream applications such as yield estimation, phenotype-genotype association studies, and resource allocation.

The multidisciplinary implications of LKNet extend well beyond academic novelty. By delivering robust and high-throughput panicle counting under a spectrum of phenotypic and environmental variabilities, this model paves the way for scalable UAV-based crop monitoring systems that require minimal manual annotation efforts. This attribute accelerates phenotyping pipelines and cultivates the potential for real-time, field-based decision support in breeding programs and precision farming.

Innovations embedded within LKNet also represent a template for next-generation computer vision methodologies tailored to agriculture. Its dynamic receptive field adaptation and flexible loss formulation address the quintessential challenge of translating image-based predictions into actionable agronomic insights—a longstanding barrier in applying AI to complex biological systems. As such, LKNet exemplifies the ongoing convergence of deep learning, remote sensing, and crop science, signaling a transformative shift toward data-driven agricultural sustainability.

Future directions may explore integrating LKNet with multimodal sensor data, including hyperspectral imaging and environmental metadata, to further refine phenotyping accuracy and contextual understanding. Additionally, scaling the approach across other crop species and geographic zones will be critical to validate generalizability and encourage widespread adoption.

In sum, LKNet stands as a landmark advancement in crop phenotyping technology. By leveraging architectural ingenuity and rigorous validation, it delivers exceptional counting precision and operational efficiency across diverse rice canopy conditions. Emerging from this work is a compelling vision for intelligently automated, UAV-enabled crop monitoring systems poised to revolutionize agricultural research and practice.

Subject of Research: Not applicable

Article Title: LKNet: Enhancing rice canopy panicle counting accuracy with an optimized point-based framework

News Publication Date: 28-Feb-2025

References:
10.1016/j.plaphe.2025.100003

Keywords:
Plant sciences, Technology, Agriculture