Deep Learning Reads 3D Point Clouds to Detect and Quantify Damage in Ancient Masonry Walls

Centuries-old masonry walls do not fail suddenly. They erode slowly, grain by grain, as frost, salt, rain and biological growth gnaw away at brick and stone surfaces, leaving scars that are easy to overlook and hard to measure. For conservators responsible for structures such as the Nanjing City Wall, one of the largest surviving ancient city walls in the world, the question of how much material has been lost, and how deeply, has long depended on manual inspection, tape measures and photographic surveys. A new study published in npj Heritage Science offers a different approach: a deep learning framework that ingests three-dimensional laser-scanned point clouds and automatically detects, maps and quantifies surface deterioration on masonry walls.

The research, led by Fan Sun, Qing Chun, Shiyu Ma and Yu Yuan of the School of Architecture at Southeast University in Nanjing, addresses a persistent bottleneck in heritage documentation. Image-based inspection methods, which have become popular because cameras are cheap and photographs are easy to collect, suffer from two fundamental weaknesses. First, they depend on illumination: shadows, glare and uneven lighting can hide damage or create false impressions of it. Second, a two-dimensional photograph loses depth information, so a shallow surface flake and a deep cavity that look identical in plan may be indistinguishable without laborious supplementary measurement. Point cloud data, captured by terrestrial laser scanning or photogrammetric reconstruction, sidestep both problems by recording dense, accurate three-dimensional geometry regardless of surface color or lighting conditions.

But raw point clouds bring their own challenge. Unlike an image, which arrives as a tidy grid of pixels, a point cloud is an unordered set of millions of XYZ coordinates with irregular spacing. Convolutional neural networks designed for photographs cannot be applied directly. The Southeast University team built their pipeline on PointNet++, a neural architecture specifically designed for point sets, which learns hierarchical features by grouping points into local neighborhoods and abstracting them layer by layer. Their contribution is an improved version of this architecture incorporating what they call a Novel Set Abstraction, or NSA, module, which refines how the network samples and aggregates geometric information from the scanned surface.

The framework operates in three integrated stages. The first is preprocessing, in which the raw scans are cleaned and prepared so that the network receives consistent input. The second is binary segmentation: the improved PointNet++ classifies every single point in the cloud as either damaged or undamaged, effectively painting a per-point diagnosis across the entire wall surface. The third stage is quantification, where morphological analysis of the segmented damage regions converts the classification into numbers that conservators actually need, namely the damaged area and the maximum depth of each deterioration zone. This last step exploits the very property that images lack, because depth is measured directly from the three-dimensional geometry of the scan.

To train and evaluate the model, the researchers assembled a dataset of 317 annotated patches collected from four wall segments at different locations and orientations along the Nanjing City Wall. This variety matters. A wall facing south weathers differently from one facing north, and damage near the base, where moisture and salt accumulate, differs from damage higher up. By sampling across locations and orientations, the dataset captures a representative range of the surface deterioration that the wall actually exhibits, giving the network a fair test of generalization within the structure it was designed to monitor.

The performance figures tell a striking story about the value of the architectural improvement. The original PointNet++ achieved an intersection over union, the standard overlap metric for segmentation tasks, of only 28.80 percent on the damage class, meaning the baseline network struggled to delineate eroded regions accurately. The improved model with the NSA module raised that figure to 55.98 percent, nearly doubling the damage-class score, and reached a mean intersection over union of 73.77 percent across all classes. In segmentation problems where damage boundaries are fuzzy and the damaged class occupies a minority of points, such gains are substantial rather than incremental.

Segmentation alone, however, does not answer the conservator’s question of how much wall has been lost. The team therefore validated the full pipeline, including the quantification stage, against 30 independently measured damaged regions. The automated framework produced a mean relative area error of 17.24 percent and a mean absolute maximum-depth error of just 6.77 millimeters. For a structure whose deterioration must be tracked over years and decades, an error of under seven millimeters in depth estimation, obtained without a single manual measurement, represents a meaningful advance in the precision of routine condition surveys.

The implications extend beyond one famous wall. Masonry architectural heritage is vast and heterogeneous, spanning city walls, temples, churches, fortifications and vernacular buildings on every continent, and the professionals responsible for it are chronically outnumbered by the fabric they must protect. A framework that turns laser scans, which many heritage agencies already collect for archival purposes, into quantitative damage maps could transform those archives from static records into active monitoring tools. Repeated scans of the same wall, processed by the same model, would allow deterioration rates to be compared over time, revealing which sections are stable and which are actively degrading, and helping prioritize expensive conservation interventions where they are needed most.

The authors are appropriately measured about the limits of their results. The framework was evaluated under the tested conditions of the Nanjing City Wall, and they note that it supports preliminary documentation and monitoring of surface material deterioration there, while its transferability to other structures and environments remains to be validated. That caution is scientifically sound. Deep learning models trained on one building’s materials, weathering patterns and scanning conditions cannot be assumed to perform equally on a sandstone cathedral in a different climate, and the researchers’ explicit acknowledgment of this boundary sets a clear agenda for future work on cross-site generalization.

Even with that caveat, the study demonstrates a convincing convergence of two technologies that heritage science has been pursuing in parallel for years. Three-dimensional scanning provides the geometric fidelity that photographs cannot, and deep learning provides the automation that manual annotation cannot scale to. By combining an improved point-set neural network with morphology-based quantification, the Southeast University team has shown that a wall can, in effect, report its own injuries: where the surface has failed, how extensive the failure is, and how deep it reaches. As such systems mature and are tested across more sites, the slow, invisible erosion of the world’s masonry heritage may become visible, measurable and, ultimately, far easier to manage.

Subject of Research: Deep learning-based detection and quantification of surface damage in masonry architectural heritage using 3D point cloud data

Article Title: A method for detecting and quantifying damage in masonry architectural heritage based on deep learning of point cloud data

Article References: Sun, F., Chun, Q., Ma, S., & Yuan, Y. (2026). A method for detecting and quantifying damage in masonry architectural heritage based on deep learning of point cloud data. npj Heritage Science. https://doi.org/10.1038/s40494-026-03043-z

Image Credits: AI Generated

DOI: 10.1038/s40494-026-03043-z

Keywords: deep learning, point cloud, masonry heritage, Nanjing City Wall, PointNet++, damage detection, 3D segmentation, heritage conservation, laser scanning, surface deterioration, structural health monitoring, npj Heritage Science