Loading...
Understand how autonomous systems simultaneously build maps and track their position using cameras and IMUs. From classical feature-based methods like ORB-SLAM to modern deep learning approaches, master the complete Visual SLAM pipeline.
Visual Simultaneous Localization and Mapping (SLAM) is the problem of building a map of an unknown environment while simultaneously tracking the camera's position within that map. This chicken-and-egg problem --- you need a map to localize, but you need to know your location to build a map --- is solved through iterative optimization that refines both the map and trajectory together.
SLAM systems are divided into a front-end (real-time tracking, feature matching, local optimization) and a back-end (global optimization, loop closure, map management). The front-end processes each new frame in real time, estimating the camera motion relative to the previous frame. The back-end runs asynchronously, detecting when the camera revisits a previously mapped area (loop closure) and correcting accumulated drift through global optimization.
The chapter covers the full spectrum of visual SLAM: feature-based methods (ORB-SLAM) that track sparse keypoints, direct methods (LSD-SLAM) that operate on raw pixel intensities, visual-inertial approaches (VINS-Mono) that fuse camera with IMU data, and modern deep learning methods that replace hand-crafted features with learned representations.
Key topics include:
Click any topic to jump in
Front-end tracking, back-end optimization, and keyframe selection — the core SLAM architecture.
Two paradigms for front-end motion estimation
Feature-based SLAM with ORB descriptors, covisibility graphs, and bundle adjustment.
Photometric alignment on raw pixel intensities without feature extraction.
Improving robustness with additional sensors and deep learning
Tight fusion of camera and IMU measurements via preintegration.
SuperPoint and SuperGlue replace hand-crafted features with learned representations.
Closing the loop for accurate long-range mapping
NetVLAD and HLoc for recognizing previously visited places and correcting drift.
Pose graph optimization and robust verification for globally consistent maps.
This chapter is part of PixelBank Premium. Create a free account, then upgrade to read the full lesson — concepts, walkthroughs, and exercises.