Chapter 3: Mathematical Foundations

Mathematics is the foundation of all AI and machine learning. Every algorithm you encounter -- from image filters to neural networks -- relies on concepts from linear algebra, calculus, probability, statistics, and information theory. Understanding these foundations transforms you from someone who applies algorithms to someone who truly understands them.

Linear algebra provides the language: images are matrices of numbers, model parameters are vectors, and predictions are computed through matrix multiplication. When you see Y = XW + b, you are looking at how a neural network layer transforms its inputs. Understanding how matrices transform space -- through rotations, scaling, and projections -- gives you geometric intuition for what models actually do, whether in computer vision or machine learning.

Probability and statistics help you reason about uncertainty. Models don't make perfect predictions -- they estimate probabilities. Understanding distributions, expectations, and Bayes' theorem explains why certain loss functions work and how models quantify confidence. Maximum likelihood estimation connects probability to optimization, forming the basis of most training procedures.

Information theory, developed by Claude Shannon, provides the mathematical framework for measuring information content. Concepts like entropy, cross-entropy, and KL divergence are fundamental to understanding classification loss functions, generative models, and representation learning.

Calculus, specifically gradient-based optimization, is how models learn. The derivative tells us which direction improves predictions, and optimization algorithms follow this gradient to find the best parameters. The chain rule enables backpropagation in neural networks. Understanding convexity helps explain when optimization is easy versus hard.

Matrix decompositions like Singular Value Decomposition (SVD) are workhorses of both CV and ML -- powering image compression, recommender systems, dimensionality reduction, and understanding the geometry of learned representations.

Finally, eigendecomposition underlies dimensionality reduction. Principal Component Analysis (PCA) finds directions of maximum variance by computing eigenvectors of the covariance matrix, while eigenvalues of the Hessian reveal curvature of loss landscapes. In computer vision, eigenvalues power Harris corner detection, and eigenvectors form the basis of Eigenfaces.

This chapter provides the mathematical toolkit you will use throughout your journey:

• Vectors & Vector Operations: The data structures and operations at the heart of all CV and ML
• Matrices & Transformations: How matrices represent and compose geometric transformations
• Probability & Statistics: Tools for reasoning about uncertainty and measuring model performance
• Information Theory: Entropy, cross-entropy, and KL divergence for loss functions
• Calculus for Optimization: Gradients, gradient descent, and convexity for training models
• Matrix Decompositions: SVD and its applications
• Eigenvalues & PCA: Understanding and implementing dimensionality reduction