Amodal completion enables 3D wheat reconstruction from a single image

Occlusion has long been the stubborn enemy of three-dimensional plant phenotyping. In a wheat field or even in a controlled indoor growth chamber, spikes overlap leaves, leaves overlap stems, and neighboring plants crowd one another into a tangle of partially hidden structures. For researchers trying to reconstruct complete 3D models of wheat plants from photographs, this means that a single camera view typically captures only fragments of the target. A new study published in Plant Methods now offers a way around this bottleneck by teaching computers to imagine what they cannot see, combining amodal completion—the ability to infer the full shape of an object even when parts of it are hidden—with state-of-the-art single-view 3D reconstruction models. The result is a sequential framework that first fills in the missing visual information and then converts the completed images into full 3D wheat structures.

The research, led by Wentao Song, He Huang, Fang Qu, Caiguo Tang, Youqiang Sun, and Junqing Zhang of the Hefei Institutes of Physical Science at the Chinese Academy of Sciences, together with collaborators at the University of Science and Technology of China and the Institute of Hefei Artificial Intelligence Breeding Accelerator, addresses a problem that has limited the accuracy of high-throughput phenotyping pipelines for years. When only partial regions of a wheat spike are visible in an image, downstream measurements such as spike length, width, and area are systematically biased. Traditional multi-view approaches can mitigate this by capturing the plant from many angles, but they demand expensive hardware, complex calibration, and time-consuming acquisition procedures that are impractical for large-scale breeding programs.

The new framework takes a fundamentally different approach. Rather than requiring multiple cameras or elaborate scanning rigs, it starts with a single image and applies a visible-prompt-driven amodal completion model to recover the complete appearance and structural cues of occluded targets. In computer vision terms, amodal completion refers to the estimation of an object’s full extent, including regions that are not directly observable because they are covered by other objects or fall outside the visible silhouette. The completion model in this study, Pix2Gestalt, uses visible portions of the wheat plant as prompts to synthesize plausible completions of the hidden parts, producing an image in which the target appears whole. This completed image is then fed into a single-view 3D reconstruction model that generates a full 3D structure of the plant organ.

A critical prerequisite for developing and validating such a system is data, and here the team made a substantial contribution of its own. They constructed the MMWO dataset—Multi-view Multi-instance Wheat Occlusion—derived from the MMW dataset captured under controlled indoor scenarios. Because real-world occlusion is difficult to annotate precisely, the researchers synthesized occlusion samples artificially through a carefully designed pipeline. Organ-level cutouts were extracted from the source imagery, subjected to random geometric transformations, and pasted back onto images within region constraints that mimic natural occlusion patterns. Each synthesized sample comes with rich annotations, including visible masks that delineate what can actually be seen, occlusion masks that mark the hidden regions, and complete target images that serve as ground truth for training and evaluation. This synthetic strategy gives the community a benchmark with exact knowledge of what is hidden and what is not, something nearly impossible to obtain from field photography alone.

With this dataset in hand, the researchers systematically evaluated six representative single-view 3D reconstruction methods: Direct3D, Real3D, SF3D, Spar3D, TRELLIS.2, and Hunyuan3D. These models represent the current frontier of 3D generative artificial intelligence, and applying them to wheat phenotyping is among the first systematic tests of their agricultural potential. The evaluation used complementary metrics: Chamfer Distance in its L1 and L2 variants (CD-L1 and CD-L2) to measure geometric fidelity between the reconstructed and reference 3D shapes, and the F-score as a balanced measure of precision and recall in the reconstructed geometry. Rendering quality was assessed with peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and the learned perceptual image patch similarity metric (LPIPS), which captures how visually similar rendered images appear to human observers and neural networks alike.

The results were revealing. Hunyuan3D achieved the best geometric performance among the six methods, recording the lowest mean CD-L1 and CD-L2 values of 0.1286 and 0.0536 respectively, along with the highest mean F-score of 0.5668. In other words, its generated 3D wheat structures were geometrically closest to the ground-truth targets. SF3D, by contrast, excelled in rendering quality, achieving the best scores on PSNR, SSIM, and LPIPS. This divergence between geometric accuracy and rendering fidelity is an important practical finding: a model that produces visually convincing images does not necessarily produce the most dimensionally accurate 3D geometry, and phenotyping applications that depend on precise measurements must weigh these trade-offs carefully when selecting a reconstruction backbone.

Perhaps the most striking evidence for the value of amodal completion came from the phenotypic measurements themselves. Without completion, the estimation errors for spike length, width, and area stood at 9.31 percent, 10.89 percent, and 32.23 percent respectively. The area error in particular—over thirty percent—illustrates just how severely occlusion distorts two-dimensional measurements derived from partially visible spikes. After applying Pix2Gestalt completion to the images before reconstruction and measurement, those errors dropped to 4.64 percent, 9.70 percent, and 9.45 percent. The improvement in spike area estimation is dramatic, cutting the error by more than two-thirds, while length estimation errors were roughly halved. These numbers demonstrate that filling in the invisible portions of a plant image is not merely an aesthetic enhancement but a substantive correction that propagates through the entire measurement pipeline.

The technical logic behind this improvement is worth unpacking. Single-view 3D reconstruction models learn strong priors about object shape from large training corpora, but when the input image contains a truncated or occluded target, the model receives misleading evidence about the object’s true extent. The reconstruction network may either hallucinate arbitrary geometry in the missing regions or, worse, faithfully reproduce the truncation as if the object genuinely ended where the occlusion begins. By intervening upstream with amodal completion, the framework provides the reconstruction model with a structurally complete prior. The visible-prompt mechanism ensures that the completion remains anchored to what is actually observed, so the synthesized hidden regions are constrained by the genuine appearance of the target rather than being generated in a vacuum.

The implications extend well beyond wheat. The authors position the work as a systematic reference for applying 3D generative models to agricultural phenotyping in general, and the framework’s architecture—completion followed by reconstruction—is agnostic to the specific crop, provided suitable completion and reconstruction models are available. For breeding programs that must screen thousands of genotypes for traits such as spike size and architecture, the ability to extract reliable 3D measurements from single photographs could dramatically reduce equipment costs and throughput barriers. Spike morphology is closely tied to grain number and yield, making accurate, non-destructive spike phenotyping a high-value target in crop improvement.

The study also offers a candid view of the current state of 3D generative modeling in plant science. The variation in performance across the six tested methods, and the divergence between geometric and rendering quality, signal that no single off-the-shelf model is yet ideal for phenotyping workloads. Benchmark resources like MMWO, with their exhaustive annotations of visible and occluded regions, give researchers the tools to measure progress systematically as generative 3D technology matures. The synthetic occlusion pipeline itself—cutouts, transformations, and constrained pasting—could be adapted to other crops and organ types, multiplying the utility of existing multi-view datasets by enriching them with occlusion scenarios.

This research was supported by the Anhui Provincial-Level Scientific and Technological Innovation Tackling Key Problems Plan Project, the Anhui Provincial Key Research and Development Project, and the Institute of Hefei Artificial Intelligence Breeding Accelerator. Published open access, the work invites plant scientists and computer vision researchers alike to build upon its framework. As the authors note, the study offers a feasible solution for 3D wheat phenotyping under occlusion, and with generative 3D models advancing at a remarkable pace, the marriage of amodal completion and single-view reconstruction may soon become a standard component of the digital agriculture toolbox—turning what a camera cannot see into data that breeders can use.