The study, published in the journal Cluster Computing, set out to answer a deceptively simple question: can ordinary people with a smartphone and a free app produce 3D scenes good enough for the metaverse, without any of the specialized artistic and technical skills that professional modeling demands? To find out, the researchers reconstructed the same three real-world environments three times over—once by hand in the open-source modeling software Blender, and twice using mobile photogrammetry applications. The first app, Polycam, relies on a classical computer-vision technique called Multi-View Stereo, or MVS. The second, LumaAI, is built on Neural Radiance Fields, or NeRF, a deep-learning approach that has electrified the graphics community in recent years.

The two photogrammetry techniques could hardly be more different under the hood. MVS works like digital triangulation on steroids: it detects visual features in overlapping photographs, matches them across views to estimate where the camera stood for each shot, and then triangulates the matched points into a dense cloud from which a solid polygonal mesh is extracted. Textures are generated by projecting the original photos back onto that mesh. NeRF takes an entirely different route. Instead of building geometry directly, it trains a neural network to encode the scene itself, learning to map every point in three-dimensional space, viewed from any direction, to a color and a density. The result is a continuous volumetric model of how light behaves in the scene, which can produce startlingly photorealistic reconstructions of complex surfaces and lighting effects.

For the experiment, the team selected three environments chosen to reflect the kinds of spaces metaverse developers actually want to build: a domestic living room, a working office, and a social meeting hall. Each was filled with objects varying in size, shape, and surface properties, from coffee tables and plants to desks, documents, and stationery. A single experimenter with professional 3D modeling expertise carried out all reconstructions, first photographing each room for reference and then either modeling everything from scratch in Blender or scanning it with a smartphone. After a pilot run in a kitchen to fine-tune the procedure, the reconstructed assets were assembled into coherent scenes and exported to Unity3D, a widely used game engine, where each scene was explored in real time with a first-person camera for five minutes per trial.

The headline finding concerns time, and it is dramatic. On average, manually modeled scenes took roughly 7 hours and 25 minutes to complete, while the photogrammetry pipelines—fed by the same captured videos—needed only about 2 hours and 12 minutes. Automated reconstruction cut the modeling workload by more than two-thirds. Intriguingly, the time required for the automated approach barely varied with the complexity of the scene, because filming an object takes roughly the same effort whether it is ornate or plain, whereas manual modeling time scaled directly with the number of objects and their level of detail.

Speed, however, came at a cost measured in polygons. Manual modeling consistently produced the leanest meshes, ranging from just 8,991 triangles in the meeting hall to 126,307 in the office, reflecting the clean, deliberate topology a human modeler builds by manipulating vertices and faces from primitive shapes. LumaAI, true to its NeRF origins, generated extraordinarily dense meshes exceeding 2.8 million polygons in every scene and peaking above 4 million in the living room. Polycam landed in between, producing between roughly 74,000 and 534,000 polygons thanks to an internal optimization mechanism that trims mesh density while preserving essential detail. That middle ground matters enormously: polygon count is a direct proxy for the computational burden a scene places on a graphics card, and the metaverse lives or dies on real-time rendering.

The performance tests made that burden vivid. On an NVIDIA RTX 2080, manual and Polycam scenes held acceptable frame rates of around 55 to 58 frames per second, while LumaAI scenes collapsed to roughly 30 to 32 FPS with GPU usage climbing above 80 percent. The mid-tier RTX 3060 handled manual and Polycam content comfortably at 60 FPS with GPU loads between 32 and 39 percent, but strained under LumaAI models, which pushed GPU usage toward 80 percent and dropped frame rates to the mid-thirties. Only the flagship RTX 4090 rendered everything effortlessly, sustaining 115 to 120 FPS on the lighter scenes and still above 90 FPS even for the dense NeRF meshes. The lesson is stark: without serious post-processing, NeRF-derived assets remain effectively out of reach for VR headsets and mobile platforms, precisely the devices the metaverse is meant to serve.

Visual fidelity told a more nuanced story. The researchers compared rendered images of each virtual scene against photographs of the real rooms using the Structural Similarity Index, a metric that scores similarity on a scale from minus one to one. No single technique dominated. In the cluttered living room, Polycam achieved the highest average similarity at roughly 0.58, while LumaAI performed worst despite its geometric richness. In the office, Polycam and manual modeling were statistically comparable, and in the simpler meeting hall, the hand-built scene actually won, likely because its plain object shapes made manual modeling and texture mapping especially effective. The finding that LumaAI’s hyper-detailed meshes did not translate into superior visual similarity suggests weaknesses in texture generation or lighting reconstruction rather than geometry. A complementary measure, the Hausdorff distance, quantified how far the two automated tools diverged from each other object by object: differences were large for complex, occluded items like lamps, sofas, and curtains, but fell below 0.3—and sometimes below 0.1—for simple, regular shapes such as chairs and cabinets.

Perhaps the most practically valuable insights concern which objects suit which technique. Photogrammetry excelled at small, intricate items—plants, telephones, keyboards—whose compact size allowed full-angle coverage and whose rich textures gave the algorithms abundant visual cues. It stumbled on large objects that demand more physical space to scan, on reflective surfaces whose changing highlights confuse frame-to-frame matching, and, counterintuitively, on flat, featureless furniture like tables and wardrobes, where the lack of shading variation starves the reconstruction algorithms of information. A human modeler, by contrast, can mentally reconstruct occluded geometry, ignore misleading reflections, and assemble simple shapes in minutes from basic primitives. The choice between the two apps followed similar logic: LumaAI delivered more complete, higher-fidelity meshes and handled mixed-surface objects better, while Polycam proved more robust with glossy, reflective materials and produced lighter models friendlier to constrained hardware.

So can AI-powered scanning replace the 3D artist? The researchers’ answer is an unambiguous no, at least for now. Photogrammetry is a genuine accelerant and a democratizing force, opening scene creation to people without modeling expertise and slashing prototyping time, but it surrenders control over mesh topology, produces dense and sometimes flawed geometry, and struggles with entire categories of materials. Rather than a replacement, the authors argue, these tools should be treated as complements to manual craft—ideal for rapid prototyping and complex organic objects, but still dependent on human optimization for performance-critical, professional-grade assets. Looking ahead, the team points to lightweight NeRF variants and emerging techniques like Gaussian Splatting, automated post-processing pipelines for mesh simplification and artifact correction, and human-centered studies measuring perceived realism and immersion as the next frontiers. For a metaverse that promises millions of simultaneous users in lifelike worlds, the message of this study is both cautionary and constructive: the machines are getting fast, but the artists are not obsolete yet.

Subject of Research: Comparative evaluation of manual 3D modeling and AI-based photogrammetry techniques for metaverse scene generation

Article Title: Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques

Article References: Pentangelo, V., Di Dario, D., De Martino, V., Buono, M. D., & Lambiase, S. (2026). Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques. Cluster Computing, 29(12), Article 715. https://doi.org/10.1007/s10586-026-06530-x

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06530-x

Keywords: metaverse, photogrammetry, 3D scene generation, NeRF, Multi-View Stereo, Polycam, LumaAI, real-time rendering, virtual environments, 3D modeling, GPU performance, artificial intelligence

The central innovation of the framework is that it refuses to treat reconstruction and recognition as separate problems. Most existing pipelines in this space fall into two camps. On one side sit techniques rooted in Neural Radiance Fields, or NeRF, the celebrated deep-learning method that represents a scene as a continuous volumetric function and synthesizes novel views by predicting color and density along camera rays. NeRF excels at producing photorealistic renderings but does not, by itself, tell a farmer where residues are dense or thin. On the other side sit deep learning segmentation systems such as YOLOv8, which can label pixels in individual images with impressive speed but produce only flat, two-dimensional outputs. Traditional pipelines run these tools independently and then attempt to reconcile them after the fact, a strategy the authors argue fundamentally limits residue-specific analysis.

The new framework closes that gap by injecting YOLOv8-derived semantic masks directly into the optimization process of NeuS, a neural implicit surface reconstruction method that learns signed distance functions through volume rendering. During training, the reconstruction network is guided by the segmentation evidence, so the surfaces it carves out of the scene are shaped, in part, by knowledge of where residues have been detected in the input photographs. This residue-aware integration strategy is the first of three contributions the authors highlight. Rather than reconstructing everything uniformly and asking questions later, the system concentrates its modeling capacity on the structures that matter for residue monitoring, which is essential in agricultural scenes where residues are structurally sparse and their density patterns are highly heterogeneous.

The second major contribution is a new quantitative bridge between two dimensions. The researchers introduce a metric called Coverage Density, denoted D-cov, together with a voxelized density grid representation. In plain terms, the framework takes the two-dimensional segmentation masks produced for each photograph and projects them into a three-dimensional grid of voxels, the cubic building blocks of volumetric space. Each voxel accumulates evidence from all the views that observe it, ultimately yielding a spatially explicit three-dimensional residue-density map. This transformation is what elevates the system from a collection of labeled images into a genuine three-dimensional measurement tool. A land manager can inspect the density map and immediately see where residue is concentrated, where it is patchy, and where the ground is nearly bare, without walking the field on foot.

The third contribution addresses the realities of field data. Agricultural environments are not tidy laboratory scenes. Residues decompose unevenly, wind and machinery scatter material unpredictably, and tall crops cast shadows and occlusions that confuse naive reconstruction. The framework explicitly accounts for structural sparsity and heterogeneous density patterns, the authors write, so that the density estimates remain meaningful even in the messiest scenarios. This consideration is what the team believes distinguishes their approach from prior NeRF-based agricultural pipelines, which have largely focused on crop morphology analysis, orchard scene reconstruction, and plant phenotyping, treating the field canopy rather than the ground-level residue layer as the object of interest.

The experimental results, while modest by the standards of curated benchmark datasets, are notable precisely because they were achieved in unstructured, real-world field conditions. The segmentation component achieved a mean Average Precision at an Intersection-over-Union threshold of 0.5 of approximately 0.31, and a stricter mAP spanning thresholds from 0.5 to 0.95 of approximately 0.14. Those headline figures, however, mask a crucial pattern: accuracy rises dramatically in high-density residue regions, reaching roughly 0.72. In other words, the system is most reliable exactly where it matters most for residue management decisions, identifying thick concentrations of material with confidence, while remaining more cautious about ambiguous, sparse debris.

The three-dimensional outputs are the study’s most striking demonstration. The generated residue-density maps successfully captured spatial variations across representative field scenarios, with estimated density values ranging between 55.7 percent and 77.7 percent. These are not merely rendered pictures; they are quantitative spatial estimates that can be compared across fields, across seasons, or against management targets. For precision agriculture, such maps could inform decisions about how much residue to retain for soil moisture and erosion control, where tillage is needed, and how residue distribution affects planting operations. For carbon cycle science, the ability to estimate residue cover from inexpensive imagery has direct implications for tracking how much carbon agricultural soils sequester or release, a variable that currently depends on laborious ground surveys.

Cost and accessibility are central to the work’s practical appeal. Traditional methods for quantifying residue cover, from manual line-transect sampling to satellite remote sensing, either demand scarce human labor or sacrifice spatial resolution. The proposed pipeline requires only multi-view imagery, which can be captured with consumer drones or even handheld cameras, and commodity computing resources. The authors position the framework as a scalable and cost-effective solution for crop residue monitoring, particularly in resource-constrained environments where the expensive sensors and infrastructure of high-end remote sensing are out of reach. Because the entire approach is data-driven, it can in principle be retrained for new crops, residue types, and geographic regions without redesigning the underlying system.

The scientific lineage of the project traces through some of the most influential ideas in modern computer vision. The framework builds on NeRF, introduced by Mildenhall and colleagues and later recognized in Communications of the ACM, which showed that scenes could be encoded in the weights of small neural networks. It borrows NeuS from Wang and colleagues, which extended volume rendering to recover sharp, continuous surfaces rather than fuzzy density clouds. On the perception side, it leverages the YOLO family of real-time detectors, maintained by Ultralytics, and sits alongside segmentation architectures such as Mask R-CNN that have already been applied to plant counting and sizing tasks. The authors also situate their work within a rapidly growing literature applying 3D reconstruction to agriculture, including recent NeRF-based crop morphology pipelines, UAV-based orchard reconstruction with semantic segmentation, Gaussian-splatting approaches to field reconstruction, and surveys of 2D-to-3D methods in agricultural sensing.

The team has released its dataset publicly to encourage reproducibility and collaborative research, a move that could accelerate adoption across the precision agriculture community. The study’s authors describe a clear division of labor: Ayushi conceived the research, designed and implemented the methodology, performed the experiments and drafted the manuscript, while Amar Nath supervised the work, refined the research design and critically revised the paper. Both researchers emphasize that the framework is a step toward residue-aware, semantics-guided 3D modeling as a general paradigm, one in which what a machine sees and what a machine builds become the same computation. As agriculture confronts simultaneous pressures of sustainability, carbon accounting and labor scarcity, tools that extract rich three-dimensional understanding from cheap photographs may prove to be among the most consequential applications of neural rendering yet.

Subject of Research: Hybrid NeRF-based 3D reconstruction and semantic segmentation framework for crop residue density mapping in agricultural fields

Article Title: A hybrid framework for 3D reconstruction and semantic analysis using NeRF

Article References: Ayushi, A., & Nath, A. (2026). A hybrid framework for 3D reconstruction and semantic analysis using NeRF. Multimedia Tools and Applications, 85(9), Article 757. https://doi.org/10.1007/s11042-026-21899-y

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21899-y

Keywords: NeRF, NeuS, 3D reconstruction, semantic segmentation, YOLOv8, crop residue, precision agriculture, residue density, multi-view imagery, computer vision, carbon cycle, SLIET Longowal