Geometry-Aware Transformer Boosts 3D Indoor Scene Segmentation

Researchers have introduced GAMPoint, a transformer-based framework for 3D semantic segmentation of point clouds, designed to improve predictions around thin structures, object boundaries, and rare classes in cluttered indoor scenes. The work, published in PLOS One by Xinyu Mu, Xuan He, Ziqi Huang, and Ting Liu, addresses a persistent weakness in existing segmentation methods: limited use of geometric cues.

The framework builds on Point Transformer V3 and adds two components. Enhanced Relative Positional Encoding augments relative positional modeling with local surface-variation cues, strengthening geometry-aware feature learning. Conditional Multimodal Gating acts as an input-side embedding interface: XYZ coordinates supply the basic point-cloud geometry, while the gating mechanism incorporates available auxiliary attributes such as RGB colors and surface normals before the serialized encoder.

The authors note a dataset-specific detail: for RGB-D reconstructed datasets such as ScanNet, the XYZ coordinates already contain depth-derived geometric information, so raw depth is not added as a separate per-point channel. This multimodal-ready design allows the model to flexibly use whichever auxiliary attributes a dataset provides.

GAMPoint was evaluated on two widely used indoor benchmarks, S3DIS and ScanNet. Experimental results show competitive performance and modest gains over strong point-transformer baselines. The reported improvements are not uniform across all categories: class-wise analyses indicate category-dependent improvements, meaning gains vary by object class rather than applying equally everywhere.

To assess reliability, the team also ran repeated-run analyses under the reported settings. These showed consistent gains across matched repeated runs, suggesting the improvements are stable rather than artifacts of a single training run. The scope remains indoor scene segmentation; the study does not report results on outdoor or other 3D domains.

The code has been released at github.com/airmanss/GAMPoint, allowing other researchers to reproduce the reported settings and extend the multimodal interface to additional attribute types. Overall, the study positions geometry-aware positional encoding combined with conditional multimodal gating as a practical route to more accurate segmentation of fine structures and underrepresented classes in indoor point clouds.

Subject of Research: A geometry-augmented vision transformer framework for 3D point-cloud semantic segmentation of indoor scenes

Article Title: GAMPoint: Geometry-augmented vision transformer with a multimodal-ready interface for 3D point-cloud segmentation

Article References: Mu, X., He, X., Huang, Z., & Liu, T. (2026). GAMPoint: Geometry-augmented vision transformer with a multimodal-ready interface for 3D point-cloud segmentation. PLOS One, 21(10), e0360907. https://doi.org/10.1371/journal.pone.0360907

Image Credits: AI Generated

DOI: 10.1371/journal.pone.0360907

Keywords: 3D point clouds, semantic segmentation, vision transformer, geometry-aware learning, multimodal fusion, S3DIS, ScanNet, indoor scenes, positional encoding, deep learning, GAMPoint, Geometry-augmented