New Dual-Stream Training Strategy Sharpens 3D LiDAR Segmentation for Autonomous Driving

Self-driving cars, delivery robots, and urban mapping platforms all depend on the same fundamental perceptual act: making sense of the millions of laser reflections that a LiDAR sensor scatters across the world every second. Turning that raw swarm of three-dimensional points into labeled categories—road, car, pedestrian, vegetation—is the task of semantic segmentation, and it remains one of the most demanding problems in 3D scene understanding. A team of researchers at Shenzhen Technology University, led by Ye Gu and Lixin Liang of the School of Artificial Intelligence together with Gengliang Chen of the Sino-German College of Intelligent Manufacturing, now reports a new framework that pushes the accuracy of LiDAR segmentation to a mean intersection-over-union of 74.4 percent on the SemanticKITTI benchmark and 83.5 percent on nuScenes, two of the most widely used datasets in the field. The work, published open access in Complex & Intelligent Systems, combines an unusually expressive convolutional backbone with a training-time trick that borrows knowledge from other sensor modalities without slowing the network down at inference.

The first half of the innovation lies in the backbone architecture, which the authors call a 3D large sparse multi-directional convolution. To understand why this matters, it helps to recall how modern LiDAR networks process point clouds. Raw LiDAR data is irregular: each laser return is a point floating in space, with no grid structure that a conventional convolutional neural network can slide over. The standard solution is voxelization—dividing space into small cubic cells and treating occupied cells as sparse entries in a three-dimensional tensor. Sparse convolutions then operate only on the occupied cells, which keeps computation tractable even when a single scan contains well over a hundred thousand points.

The catch is that ordinary sparse convolutions use small kernels, typically limited to a few neighboring voxels. That gives each layer a narrow receptive field, so the network must stack many layers before any single voxel can “see” enough surrounding context to decide whether it belongs to a curb, a wall, or a distant vehicle. The Shenzhen team attacks this bottleneck with dynamic large sparse kernels, which allow the convolutional footprint to grow dramatically while still operating only on the sparse set of occupied voxels. Because the kernel is dynamic, its shape can adapt to the local geometry of the scene rather than remaining fixed, letting the network concentrate its receptive field along structures such as elongated roads or the flat planes of building facades.

Large kernels alone, however, can still miss context that does not align with the kernel’s orientation. The second ingredient of the backbone, multi-directional convolution, addresses this by sweeping the receptive field along several directions at once. The combination creates what the authors describe as large effective receptive fields: regions of the scene over which a single computational unit can integrate evidence. In practice, this means that a voxel at the edge of a scan can draw on information from far across the street, which is precisely the kind of long-range context needed to resolve ambiguous geometry in outdoor driving scenes.

Before any of this convolution happens, the raw points must be organized, and here the framework introduces a further refinement called radially non-uniform cylindrical voxelization. Standard cubic voxelization treats all regions of space equally, but LiDAR sensors do not sample the world equally. Points near the sensor are dense, while points at the edge of the sensor’s range are spread thinly, so a uniform grid leaves near-field cells crowded and far-field cells nearly empty. Cylindrical voxelization, organized around the sensor’s axis, naturally matches the geometry of the scan, and making the radial divisions non-uniform—finer near the sensor, coarser farther out—balances the number of points per voxel across the entire scene. The result is a more even representation that prevents the network from being dominated by nearby structures and helps it learn features that transfer across distances.

The second half of the framework is where the approach becomes genuinely distinctive. During training, the 3D backbone is accompanied by two auxiliary streams: one that processes 2D RGB images, and one that processes the point cloud after it has been serialized into a 1D sequence using a space-filling curve. Space-filling curves, such as the curves that wind through a volume while visiting neighboring cells consecutively, flatten a 3D structure into an ordered sequence that preserves local adjacency, allowing lightweight 1D networks to extract patterns from the point cloud. The RGB stream, meanwhile, captures the appearance information that cameras provide but LiDAR cannot—texture, color, and fine visual boundaries.

Knowledge from these two auxiliary modalities is then distilled into the 3D backbone through a multi-layer bidirectional polarity-aware linear cross-attention mechanism. Cross-attention lets each stream query the others, so features from the image stream and the serialized point stream can inform the features being learned by the 3D convolutional network. The bidirectional design allows information to flow in both directions between streams, and the polarity-aware component distinguishes the sign or orientation of the features being aligned, which helps the mechanism match corresponding structures across modalities that represent the world in very different ways. Crucially, all of this exchange happens only during training. Once the network has been trained, the auxiliary streams are discarded, and the deployed model is the 3D backbone alone—meaning the extra accuracy gained from images and serialized point sequences comes at zero additional inference cost.

This zero-cost distillation is the aspect of the work with the clearest practical significance. Real-time perception systems on autonomous vehicles operate under strict computational budgets, and any module that must run on board—however accurate—must also be fast. Techniques that fuse camera and LiDAR features at inference time can improve accuracy but add latency and hardware complexity. By confining the multi-modal interaction to the training phase, the Shenzhen framework captures much of the benefit of multi-modal learning while keeping the deployed model as lean as a single-modality network. For engineering teams weighing the cost of additional sensors and compute on a production vehicle, that distinction could be decisive.

The reported benchmark numbers place the framework among the strongest results on both datasets. SemanticKITTI, collected in Karlsruhe, Germany, is the canonical benchmark for outdoor LiDAR segmentation, with sequences spanning tens of kilometers of urban and highway driving; nuScenes, developed for autonomous driving research, pairs LiDAR with camera, radar, and other sensors across a thousand scenes in Singapore and Boston. Mean intersection-over-union, the standard accuracy metric, measures the overlap between predicted and ground-truth labels for each class and averages across them, so gains of even a few points are meaningful. Achieving 74.4 percent on SemanticKITTI and 83.5 percent on nuScenes reflects the combined effect of the large receptive fields, the balanced voxelization, and the dual-stream distillation.

The research was supported by the Shenzhen Science and Technology Program under grants JCYJ20220818102215034 and 20231129112637001, and in part by the National Key Research and Development Program of China under grant 2024YFB4709503. The article was received on 15 June 2026, accepted on 7 September 2026, and published on 28 September 2026 under a Creative Commons license that permits non-commercial sharing with attribution. As with any benchmark-driven advance, the ultimate test will be how the framework transfers to the messier conditions of production deployments—different sensor configurations, adverse weather, and novel cities—but the paper offers a clear demonstration that a 3D network can be taught by its richer siblings during training and still run solo, and fast, once it hits the road.

Subject of Research: Knowledge distillation and large sparse multi-directional convolution for 3D LiDAR point cloud semantic segmentation

Article Title: Dual stream knowledge distillation for 3D LiDAR segmentation model with large sparse multi-directional convolution

Article References: Gu, Y., Xiao, H., Chen, G., & Liang, L. (2026). Dual stream knowledge distillation for 3D LiDAR segmentation model with large sparse multi-directional convolution. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-026-02523-w

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02523-w

Keywords: LiDAR segmentation, point clouds, knowledge distillation, sparse convolution, autonomous driving, SemanticKITTI, nuScenes, 3D scene understanding, cross-attention, voxelization, computer vision, deep learning