The study, published in the journal Cluster Computing, set out to answer a deceptively simple question: can ordinary people with a smartphone and a free app produce 3D scenes good enough for the metaverse, without any of the specialized artistic and technical skills that professional modeling demands? To find out, the researchers reconstructed the same three real-world environments three times over—once by hand in the open-source modeling software Blender, and twice using mobile photogrammetry applications. The first app, Polycam, relies on a classical computer-vision technique called Multi-View Stereo, or MVS. The second, LumaAI, is built on Neural Radiance Fields, or NeRF, a deep-learning approach that has electrified the graphics community in recent years.

The two photogrammetry techniques could hardly be more different under the hood. MVS works like digital triangulation on steroids: it detects visual features in overlapping photographs, matches them across views to estimate where the camera stood for each shot, and then triangulates the matched points into a dense cloud from which a solid polygonal mesh is extracted. Textures are generated by projecting the original photos back onto that mesh. NeRF takes an entirely different route. Instead of building geometry directly, it trains a neural network to encode the scene itself, learning to map every point in three-dimensional space, viewed from any direction, to a color and a density. The result is a continuous volumetric model of how light behaves in the scene, which can produce startlingly photorealistic reconstructions of complex surfaces and lighting effects.

For the experiment, the team selected three environments chosen to reflect the kinds of spaces metaverse developers actually want to build: a domestic living room, a working office, and a social meeting hall. Each was filled with objects varying in size, shape, and surface properties, from coffee tables and plants to desks, documents, and stationery. A single experimenter with professional 3D modeling expertise carried out all reconstructions, first photographing each room for reference and then either modeling everything from scratch in Blender or scanning it with a smartphone. After a pilot run in a kitchen to fine-tune the procedure, the reconstructed assets were assembled into coherent scenes and exported to Unity3D, a widely used game engine, where each scene was explored in real time with a first-person camera for five minutes per trial.

The headline finding concerns time, and it is dramatic. On average, manually modeled scenes took roughly 7 hours and 25 minutes to complete, while the photogrammetry pipelines—fed by the same captured videos—needed only about 2 hours and 12 minutes. Automated reconstruction cut the modeling workload by more than two-thirds. Intriguingly, the time required for the automated approach barely varied with the complexity of the scene, because filming an object takes roughly the same effort whether it is ornate or plain, whereas manual modeling time scaled directly with the number of objects and their level of detail.

Speed, however, came at a cost measured in polygons. Manual modeling consistently produced the leanest meshes, ranging from just 8,991 triangles in the meeting hall to 126,307 in the office, reflecting the clean, deliberate topology a human modeler builds by manipulating vertices and faces from primitive shapes. LumaAI, true to its NeRF origins, generated extraordinarily dense meshes exceeding 2.8 million polygons in every scene and peaking above 4 million in the living room. Polycam landed in between, producing between roughly 74,000 and 534,000 polygons thanks to an internal optimization mechanism that trims mesh density while preserving essential detail. That middle ground matters enormously: polygon count is a direct proxy for the computational burden a scene places on a graphics card, and the metaverse lives or dies on real-time rendering.

The performance tests made that burden vivid. On an NVIDIA RTX 2080, manual and Polycam scenes held acceptable frame rates of around 55 to 58 frames per second, while LumaAI scenes collapsed to roughly 30 to 32 FPS with GPU usage climbing above 80 percent. The mid-tier RTX 3060 handled manual and Polycam content comfortably at 60 FPS with GPU loads between 32 and 39 percent, but strained under LumaAI models, which pushed GPU usage toward 80 percent and dropped frame rates to the mid-thirties. Only the flagship RTX 4090 rendered everything effortlessly, sustaining 115 to 120 FPS on the lighter scenes and still above 90 FPS even for the dense NeRF meshes. The lesson is stark: without serious post-processing, NeRF-derived assets remain effectively out of reach for VR headsets and mobile platforms, precisely the devices the metaverse is meant to serve.

Visual fidelity told a more nuanced story. The researchers compared rendered images of each virtual scene against photographs of the real rooms using the Structural Similarity Index, a metric that scores similarity on a scale from minus one to one. No single technique dominated. In the cluttered living room, Polycam achieved the highest average similarity at roughly 0.58, while LumaAI performed worst despite its geometric richness. In the office, Polycam and manual modeling were statistically comparable, and in the simpler meeting hall, the hand-built scene actually won, likely because its plain object shapes made manual modeling and texture mapping especially effective. The finding that LumaAI’s hyper-detailed meshes did not translate into superior visual similarity suggests weaknesses in texture generation or lighting reconstruction rather than geometry. A complementary measure, the Hausdorff distance, quantified how far the two automated tools diverged from each other object by object: differences were large for complex, occluded items like lamps, sofas, and curtains, but fell below 0.3—and sometimes below 0.1—for simple, regular shapes such as chairs and cabinets.

Perhaps the most practically valuable insights concern which objects suit which technique. Photogrammetry excelled at small, intricate items—plants, telephones, keyboards—whose compact size allowed full-angle coverage and whose rich textures gave the algorithms abundant visual cues. It stumbled on large objects that demand more physical space to scan, on reflective surfaces whose changing highlights confuse frame-to-frame matching, and, counterintuitively, on flat, featureless furniture like tables and wardrobes, where the lack of shading variation starves the reconstruction algorithms of information. A human modeler, by contrast, can mentally reconstruct occluded geometry, ignore misleading reflections, and assemble simple shapes in minutes from basic primitives. The choice between the two apps followed similar logic: LumaAI delivered more complete, higher-fidelity meshes and handled mixed-surface objects better, while Polycam proved more robust with glossy, reflective materials and produced lighter models friendlier to constrained hardware.

So can AI-powered scanning replace the 3D artist? The researchers’ answer is an unambiguous no, at least for now. Photogrammetry is a genuine accelerant and a democratizing force, opening scene creation to people without modeling expertise and slashing prototyping time, but it surrenders control over mesh topology, produces dense and sometimes flawed geometry, and struggles with entire categories of materials. Rather than a replacement, the authors argue, these tools should be treated as complements to manual craft—ideal for rapid prototyping and complex organic objects, but still dependent on human optimization for performance-critical, professional-grade assets. Looking ahead, the team points to lightweight NeRF variants and emerging techniques like Gaussian Splatting, automated post-processing pipelines for mesh simplification and artifact correction, and human-centered studies measuring perceived realism and immersion as the next frontiers. For a metaverse that promises millions of simultaneous users in lifelike worlds, the message of this study is both cautionary and constructive: the machines are getting fast, but the artists are not obsolete yet.

Subject of Research: Comparative evaluation of manual 3D modeling and AI-based photogrammetry techniques for metaverse scene generation

Article Title: Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques

Article References: Pentangelo, V., Di Dario, D., De Martino, V., Buono, M. D., & Lambiase, S. (2026). Towards AI-powered automatic 3D scene generation for the metaverse: a comparative analysis of manual and photogrammetry techniques. Cluster Computing, 29(12), Article 715. https://doi.org/10.1007/s10586-026-06530-x

Image Credits: AI Generated

DOI: 10.1007/s10586-026-06530-x

Keywords: metaverse, photogrammetry, 3D scene generation, NeRF, Multi-View Stereo, Polycam, LumaAI, real-time rendering, virtual environments, 3D modeling, GPU performance, artificial intelligence

The team, led by Chia-Hsing Chiu and Yi-Ling Chen of the Department of Computer Science and Information Engineering, set out to address a blind spot in the current literature. Modern approaches to capturing vegetation, from image-based tree modeling to neural radiance fields and 3D Gaussian splatting, have become remarkably good at replicating lifelike geometric and topological detail. What they have not done, the authors argue, is constrain their appetite. Memory usage and rendering cost in these methods are essentially unconstrained, and no existing scenario reconstructs canopies under tight resource budgets. For interactive landscape design, where dozens of trees may populate a scene and the viewer is constantly moving, that omission is decisive.

The core idea of the new method is a deliberate act of simplification guided by human perception. Rather than reconstructing every branch and leaf, the pipeline first fits the tree’s silhouette, observed from multiple viewpoints, with semi-ellipsoids that the authors call proxies. These proxies act as lightweight stand-ins for the canopy’s bulk. Exemplar leaves are then distributed quasi-randomly across the proxy shell, producing the aggregated appearance of leaf clusters while preserving memory. The insight is that a viewer at a distance perceives the canopy as a textured surface of clustered foliage, not as a collection of individually resolvable leaves, so the shell of the proxy is the only place where detail actually matters.

The appearance model itself is built on a physically grounded decomposition of light. Following the rendering equation, the light leaving a surface point can be split into a diffuse component, which depends only on position and is view-independent, and a residual component that gathers everything directional: specular reflection, sub-surface scattering, transmission, and related effects. The authors derive this decomposition formally in an appendix, showing that the diffuse term can be bounded and estimated from input photographs by choosing, for each leaf, the camera whose viewing direction best aligns with the surface normal. This image-based texturing step stabilizes the coloring and minimizes perspective distortion during back projection.

The crucial memory saving comes from collapsing the diffuse term to a single representative radiance per leaf. Because leaves are tiny relative to the canopy, their normal variations are negligible compared with the exterior normal of the whole crown, and human perception is far more sensitive to the global color distribution of the canopy than to local color shifts within a single leaf. The team therefore assumes each leaf shares one representative normal and occlusion value, then quantizes the diffuse radiance across the entire canopy into a small set of representative index colors using K-Means clustering. Each leaf stores only its quantized color index, compressed as the product of a user-given albedo texture and the estimated light integral, rather than a high-resolution texture patch.

View-dependent subtlety is handled by the residual term, which is where the visual richness of real foliage lives: glints of light bouncing between leaves, soft variations as the canopy is seen from different angles. Because environment lighting varies smoothly and leaves are distributed chaotically and roughly evenly inside the canopy, the residual varies smoothly across the proxy surface. The authors exploit this by averaging the available per-leaf residual samples, weighting each input view by how closely its outgoing direction matches the current viewing angle, and then aggregating leaf-based residuals into proxy-based residuals normalized by the proxy normal. The result is that pixel-based multi-view subtlety textures are encoded as proxy-based light fields, a compact representation that captures how the canopy’s appearance changes as the observer moves.

The final shading model is elegantly simple: the outgoing light from any leaf is approximated as the sum of a view-independent diffuse color per leaf and a view-dependent residual lighting per proxy. In practice this means the entire canopy’s appearance is compressed into a handful of small textures, one holding quantized leaf colors and the others holding the residual light fields, while the geometry is reduced to semi-ellipsoidal shells studded with quasi-randomly placed exemplar leaves. Everything is designed to be evaluated in a single pass on a GPU, which is what makes real-time rendering feasible even for dense, visually complex canopies.

To validate the approach, the researchers conducted a series of experiments comparing their method against state-of-the-art baselines, including commercial and academic tools for tree reconstruction and rendering such as SpeedTree, Context Capture, and procedural painting approaches, alongside neural rendering techniques. The evaluation data has been made publicly available through the group’s project website, and data to reproduce the evaluation is available on request. The results show that the proposed method achieves superior memory usage and comparable rendering efficiency relative to the baselines, while presenting a visual appearance similar to the ground-truth imagery. In other words, the method matches its heavier competitors where it counts for viewers and beats them decisively where it counts for hardware.

The team also ran a user study to assess how the synthesized canopies are perceived, applying established statistical procedures for comparing ranked judgments, including Friedman-style significance testing and the method of paired comparisons, tools long used in perceptual evaluation of graphics and tone mapping. The study supports the central perceptual claim of the paper: that quantizing leaf colors and compressing view-dependent effects into proxy light fields does not produce a noticeable degradation in perceived realism for the intended viewing conditions. This perceptual grounding is what separates the work from naive level-of-detail schemes that simply swap geometry for billboards, a strategy explored in earlier vegetation rendering research dating back to slicing and blending techniques from 2000 and adaptive billboard clouds from 2014.

The implications extend across several domains. For interactive landscape and architectural design, the method offers a way to populate scenes with photorealistic trees without the memory blowup that has historically forced designers to accept crude stand-ins. For virtual and augmented reality, where rendering budgets are measured in milliseconds and megabytes, the proxy-based light field representation is a natural fit. And for the broader graphics community, the paper is a reminder that the path to realism does not always run through more data. By starting from the physics of light transport, observing what human vision actually registers, and aggressively compressing everything else, the NTUST team has shown that a dense canopy of thousands of leaves can live comfortably within a real-time budget. The work was supported by grants from Taiwan’s National Science and Technology Council, and the authors report no competing interests.

Subject of Research: Memory-efficient real-time synthesis and rendering of dense tree canopies for landscape visualization

Article Title: Real–time appearance–driven memory–efficient dense canopy synthesis

Article References: Chiu, C.-H., Lai, Y.-C., Chang, C.-W., Du, H.-Y., Tai, W.-K., & Chen, Y.-L. (2026). Real–time appearance–driven memory–efficient dense canopy synthesis. Multimedia Tools and Applications, 85(9), Article 748. https://doi.org/10.1007/s11042-026-21807-4

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21807-4

Keywords: computer graphics, tree canopy synthesis, image-based modeling, light fields, real-time rendering, memory efficiency, texture compression, plant modeling, landscape visualization, diffuse lighting, residual lighting, user study