AI Turns Public Sculptures Into Stunning Interactive 3D Digital Twins

Public art has always posed a stubborn problem for digitization. A bronze statue gleaming in a city plaza or a stainless-steel installation catching afternoon sunlight is precisely the kind of scene that defeats conventional 3D capture: reflective surfaces scramble photogrammetric matching, changing illumination shifts colors between frames, and pedestrians or vehicles wander through the shot. A new study published in Discover Artificial Intelligence by Dan Shen of Xijing University reports a reliability-aware neural radiance field workflow that confronts these obstacles head-on, achieving photorealistic, browser-ready reconstructions of five sculptures and public installations with remarkably low geometric error and smooth interactive performance.

Neural radiance fields, or NeRF, represent a scene as a continuous volumetric function: a neural network takes a 3D position and viewing direction and predicts a density and a view-dependent color at each point. Rendering proceeds by casting rays through the scene, sampling points along each ray, and accumulating their predicted colors weighted by accumulated transmittance, which naturally models occlusion and soft appearance transitions. Because the representation is implicit, NeRF can capture fine texture detail and complex lighting behavior without explicit mesh extraction, a property that makes it attractive for artworks whose carved patterns, material transitions, and light-and-shadow interplay are central to their meaning. Yet the authors emphasize that applying NeRF to outdoor public art is not straightforward, since uncontrolled lighting, partial occlusion, and uneven viewpoint distribution can inject contradictory supervision into training.

The core scientific contribution of the study is deliberately scoped: not a new radiance-field backbone, but a visibility-weighted cross-view supervision strategy. During preprocessing, background regions, moving occluders, saturated highlights, and strongly reflective areas are annotated as low-reliability zones and converted into binary masks. These masks are then propagated into a cross-view consistency loss: for corresponding pixels in two views, a reliability weight is computed as the product of the two views’ mask values, evaluated after projecting sampled points between cameras using estimated poses and ray depths. Only view pairs whose overlapping valid region exceeds 30 percent are retained, and correspondences must fall below a 2-pixel reprojection error. The weighted consistency term, combined with a local gradient agreement constraint, suppresses contradictory evidence from unreliable regions while the standard photometric rendering loss continues to supervise all valid pixels, scaled by a regularization coefficient of 0.05.

The experimental corpus comprised five purposively selected artworks spanning diverse materials and environments: an outdoor bronze sculpture, an indoor acrylic-and-metal installation, a stone relief, a stainless-steel and wood installation, and a mixed-material work. Each was captured from 65 to 120 viewpoints using three acquisition modes: handheld circular shooting for medium outdoor pieces, tripod-and-slider rigs for indoor installations and reliefs, and drone-based multi-orbit flight for large plaza works. Canon EOS cameras with high dynamic range capability handled ground photography, while DJI platforms covered aerial orbits, with all images captured at resolutions of at least 3840 by 2160 pixels. Roughly 80 percent of images trained the model and 20 percent were held out as unseen test views, separated by viewpoint order rather than random frame selection so that the test set contained both frontal and oblique observations. Crucially, every compared method used identical images, camera poses, and test views.

Preprocessing involved conventional but essential stabilization steps. Frames with severe motion blur, dominant occlusion, or unrecoverable overexposure were excluded, and channel-wise z-score and min-max normalization reduced scale differences across cameras and lighting conditions. The authors are careful to attribute no novelty to these operations; the methodological weight rests on the reliability masks and their propagation into cross-view weighting. The network itself used a dual-branch architecture separating density estimation from color regression, hierarchical coarse-to-fine sampling, and Fourier positional encoding to capture high-frequency geometric and texture detail. Training used a batch size of 2048 rays, an initial learning rate of 5 times 10 to the minus 4, the Adam optimizer, and roughly 200,000 iterations, with the color branch frozen early in training so that stable spatial structure could be learned first.

The quantitative results are striking within the study’s protocol. The proposed configuration achieved a normalized geometric error of 1.20 plus or minus 0.20 percent, compared with 3.80 plus or minus 0.38 percent for a mesh-based structure-from-motion and multi-view stereo pipeline and 2.40 plus or minus 0.26 percent for depth-map synthesis. On held-out views, the method reached a peak signal-to-noise ratio of 28.46 plus or minus 0.70 dB and a structural similarity index of 0.921 plus or minus 0.010, against 23.08 dB and 0.809 for the mesh pipeline and 25.14 dB and 0.867 for depth-map synthesis. Multi-view Recall, measuring the proportion of regions consistently recovered across eight held-out views per artwork, reached 96.4 percent versus 84.7 and 78.2 percent for the classical baselines. Exploratory paired t-tests across the five matched cases yielded p-values below 0.001 for all comparisons, though the authors stress these describe consistency within the selected cases rather than population-level superiority.

Qualitative inspection reinforced the numbers. On the stainless-steel and wood installation captured under direct afternoon sunlight, the standard NeRF baseline showed visible color diffusion and loss of edge definition at material boundaries, while the reliability-aware configuration preserved clearer separation between adjacent surfaces and produced fewer view-dependent artifacts in partially occluded regions. A 12-participant visual-expressiveness panel rated the NeRF outputs at a mean of 4.58 on a five-point scale, with lower mean squared error and higher edge sharpness than either classical pipeline. The ablation analysis traced each component’s contribution: mask guidance lifted PSNR from 27.72 to 28.05 dB, cross-view consistency regularization raised multi-view Recall from 93.2 to 95.1 percent, and adaptive rendering increased frame rate from 21 to 27 frames per second while cutting initial loading latency from 1.8 to 1.3 seconds.

The deployment layer translated the learned radiance fields into a browser-based interactive system built on WebGL and Three.js. Reconstructed outputs were reorganized into multi-resolution assets and cached through a manifest, with display level controlled by device load, recent frame time, interaction velocity, and local texture complexity. During rapid rotation or zooming the system serves coarse tiles to maintain responsiveness, then progressively refines detail once the viewpoint stabilizes. A pseudo-augmented-reality alignment mechanism matches the user’s real-time camera viewpoint with the reconstructed model’s observation vector through frustum projection, and hotspot regions trigger artwork descriptions, material annotations, and audio commentary. A 24-participant evaluation drawn from art and design education, cultural heritage administration, and urban public-space practice produced an overall rating of 4.42 plus or minus 0.34, with operational intuitiveness scoring highest at 4.58 and immersion showing the widest dispersion at 4.28.

The study is candid about its limits, which is part of what makes it a useful template for the field. The five artworks were purposively selected, so the sample is not statistically representative; the geometric reference was quality-controlled photogrammetry rather than laser-scanned metrological ground truth; and modern neural baselines such as Instant-NGP, Mip-NeRF 360, and 3D Gaussian Splatting were not executed on the same dataset, an absence the authors report openly rather than obscure behind classical comparators. The user study lacked documented order counterbalancing, and comparator-level dispersion was unavailable for inferential testing between visualization systems. Reliability masks, the authors note, reduce contradictory supervision but cannot recover surfaces that were never observed, leaving specular metal, translucent materials, and severe backlighting as open challenges.

Even with those caveats, the work signals where digital cultural heritage is heading. Museums, university digital humanities courses, and community cultural tourism projects all depend on high-quality, low-latency 3D content, and the demonstrated combination of sub-1.5-percent geometric error, 27-fps browser rendering, and 1.3-second loading suggests that NeRF-based pipelines can move from research demos toward practical exhibition infrastructure. The authors call for future evaluations that run Instant-NGP and 3D Gaussian Splatting under identical acquisition, pose, and test-view conditions, validate absolute geometry against laser scanning, and include counterbalanced, preregistered user studies. If those confirmations arrive, the reliability-aware strategy described here could become a standard ingredient in how cities preserve, share, and reinterpret the sculptures that animate their public spaces.

Subject of Research: Neural radiance field reconstruction and interactive browser visualization of public art sculptures

Article Title: High fidelity three dimensional reconstruction and interactive visualization of public art sculptures using neural radiance fields

Article References: High fidelity three dimensional reconstruction and interactive visualization of public art sculptures using neural radiance fields. (n.d.). https://doi.org/10.1007/s44163-026-02441-3

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02441-3

Keywords: neural radiance fields, 3D reconstruction, public art, sculpture digitization, interactive visualization, WebGL, Three.js, photogrammetry, cultural heritage, novel view synthesis, computer vision, digital exhibition