Chapter 8: 3D Object Detection

3D object detection is one of the most critical perception tasks in autonomous systems. Unlike 2D detection which produces bounding boxes on images, 3D detection estimates the full 3D bounding box of each object --- including its position $(x, y, z)$, dimensions $(l, w, h)$, and heading angle $\theta$ in the real world.

The primary sensor for 3D detection is LiDAR, which produces sparse point clouds with precise depth measurements. Camera-based methods are also gaining traction due to their lower cost. State-of-the-art systems typically fuse both modalities for the best accuracy.

This chapter traces the evolution of 3D detection architectures: from early voxel-based methods (VoxelNet) and efficient pillar encodings (PointPillars), through center-based detectors (CenterPoint), to modern bird's eye view (BEV) approaches that unify camera and LiDAR in a shared representation. We also cover sensor fusion strategies and the specialized evaluation metrics unique to 3D detection.

Key topics include:

• 3D bounding box parameterization and detection problem formulation
• Voxel-based and pillar-based point cloud encoding for efficient processing
• Center-based detection with heatmap predictions for anchor-free design
• Bird's eye view representations that project 3D scenes to a top-down plane
• Sensor fusion strategies for combining LiDAR and camera data
• Evaluation metrics specific to 3D detection benchmarks