Chapter 7: Point Cloud Processing
Master the algorithms and architectures for processing 3D point clouds: from fundamental operations like downsampling and normal estimation, through deep learning architectures (PointNet, PointNet++, sparse 3D convolutions), to applied tasks like segmentation, registration with ICP, and robust model fitting with RANSAC.
Chapter Overview
Point clouds are the most natural output of 3D sensors---LiDAR, depth cameras, and photogrammetry pipelines all produce unordered sets of 3D points. Processing these raw point clouds efficiently and accurately is essential for applications from autonomous driving to augmented reality.
What is this chapter about? We cover the complete point cloud processing stack. We start with fundamental operations: data structures, downsampling, normal estimation, and neighbor queries. We then explore deep learning on point clouds, beginning with PointNet's revolutionary insight that max-pooling achieves permutation invariance, continuing with PointNet++'s hierarchical feature extraction, and examining 3D convolutional approaches that voxelize the space. We apply these ideas to semantic and instance segmentation, then turn to geometric algorithms: ICP for aligning point clouds and RANSAC for fitting geometric primitives.
Why does this matter? Self-driving cars use PointNet-based models on LiDAR clouds for 3D object detection. Robotics relies on ICP for localization and RANSAC for surface fitting. Indoor mapping combines all these techniques to build navigable 3D models. Point cloud processing is where geometric algorithms meet deep learning.
How the topics connect: Point cloud fundamentals establish the data structures and basic operations. PointNet introduces the deep learning paradigm for unordered sets. PointNet++ adds local structure that PointNet misses. Voxelization and 3D convolutions offer an alternative to point-based processing. Segmentation is the key downstream task for scene understanding. ICP and RANSAC provide geometric algorithms for registration and fitting that complement learned approaches.
Chapter Roadmap
Click any topic to jump in
PC Fundamentals
Spatial data structures, voxel downsampling, and normal estimation — the building blocks of point cloud processing.
From global pooling to hierarchical local features
PointNet
The first architecture to process raw point clouds directly using per-point MLPs and symmetric max-pooling.
PointNet++
Hierarchical feature learning with set abstraction layers and multi-scale grouping for local structure.
Voxel-based networks and semantic segmentation
3D Convolutions
Voxelizing point clouds for sparse and dense 3D convolutions — MinkowskiNet and VoxelNet approaches.
PC Segmentation
Semantic, instance, and panoptic segmentation on point clouds with architectures like RandLA-Net.
Classical registration and robust model fitting
ICP Registration
Aligning point clouds via Iterative Closest Point — point-to-point, point-to-plane, and robust variants.
RANSAC 3D Fitting
Robust geometric primitive fitting — planes, spheres, and cylinders from noisy point cloud data.
Sign up to unlock this chapter
This chapter is part of PixelBank Premium. Create a free account, then upgrade to read the full lesson — concepts, walkthroughs, and exercises.