Chapter 12: 3D Gaussian Splatting
Master the revolutionary real-time radiance field technique that represents scenes as collections of 3D Gaussians. Learn how differentiable rasterization replaces expensive ray marching, how Gaussians are optimized from multi-view images, and how adaptive densification and pruning refine the scene representation. Explore extensions to dynamic scenes, 4D video, and efficient compression for deployment.
Chapter Overview
3D Gaussian Splatting (3DGS) has fundamentally changed the landscape of novel view synthesis by achieving NeRF-quality rendering at real-time frame rates. Instead of querying a neural network millions of times along camera rays, 3DGS represents the scene as an explicit collection of 3D Gaussian primitives that are projected ("splatted") onto the image plane using a fast, differentiable rasterizer.
What is this chapter about? We cover the complete 3DGS pipeline from primitive representation to optimization and rendering. You will learn how each Gaussian encodes position, shape, opacity, and view-dependent color; how the tile-based rasterizer achieves real-time performance; how gradient-based optimization fits Gaussians to multi-view images; and how adaptive densification and pruning maintain quality while controlling primitive count.
Why does this matter? 3DGS delivers the visual quality of neural radiance fields with 100-1000x faster rendering speeds. This makes it practical for real-time applications such as virtual reality, telepresence, autonomous driving simulation, and interactive content creation. Understanding 3DGS is essential for anyone working with modern 3D scene representations.
How the topics connect: We start with the Gaussian primitive representation---the atomic building block. Then differentiable rasterization explains how primitives become pixels. The optimization pipeline shows how to fit Gaussians to real images. Adaptive densification and pruning refine the representation during training. The final topics extend the framework to dynamic scenes, 4D video, and compression for practical deployment.
Chapter Roadmap
Click any topic to jump in
Gaussian Primitives
Mean, covariance, opacity, and spherical harmonics — the atomic building block of 3DGS.
Rendering and optimization form the training loop
Differentiable Rasterization
EWA splatting, tile-based rendering, and gradient flow through the rasterizer.
Optimization Pipeline
SfM initialization, photometric loss, and learning rate scheduling.
Adaptive Densification
Cloning, splitting, and pruning to refine the Gaussian representation during training.
Handling motion with deformation networks and 4D representations
Dynamic 3DGS
Deformable Gaussians and temporal deformation networks for non-static scenes.
4D Gaussians
Space-time Gaussians with temporal interpolation for dynamic scene rendering.
3DGS Compression
Pruning, quantization, and mobile deployment for compact scene representations.
Sign up to unlock this chapter
This chapter is part of PixelBank Premium. Create a free account, then upgrade to read the full lesson — concepts, walkthroughs, and exercises.