Mean Intersection over Union (mIoU)

hardPROSemantic SegmentationMetricsCV: Recognition

Implement a metric to evaluate the performance of semantic segmentation models, focusing on mean Intersection over Union (mIoU). In semantic segmentation, the goal is to assign a class label to each pixel in an image, and mIoU is a key metric for assessing the accuracy of these assignments. The concept of IoU is based on the idea of comparing the overlap between predicted and actual masks for each class, calculated as $IoU_c = \frac{TP_c}{TP_c + FP_c + FN_c}$ , where $TP_c$ , $FP_c$ , and $FN_c$ represent true positives, false positives, and false negatives for class $c$ , respectively.

To calculate IoU for each class, one must consider the intersection and union of predicted and actual masks, which can be represented as $IoU_c = \frac{|P_c \cap G_c|}{|P_c \cup G_c|}$ . The process involves:

Creating binary masks for each class in both predicted and actual labels.
Computing the intersection (logical AND) and union (logical OR) of these masks.
Calculating IoU for each class using the intersection and union.

mIoU = \frac{1}{C} \sum_{c=1}^{C} IoU_c

Here $C$ counts only the classes that appear in pred or target (union $> 0$ ). A class absent from both has an undefined IoU and is skipped, not scored as 0. If no class appears at all, return 0.0.

This technique is widely used in autonomous vehicles for scene understanding.

Example

Input

pred = [[0, 1, 1], [0, 1, 2], [2, 2, 2]]
target = [[0, 0, 1], [0, 1, 1], [2, 2, 2]]
num_classes = 3

Output

0.6389

Reasoning

pred and target are $3 \times 3$ label maps: each entry is the class index of one pixel, drawn from num_classes $= 3$ classes ( $0, 1, 2$ ).
Class 0: predicted at 2 pixels, true at 3. Both predictions are right and one true pixel is missed (top middle), so $IoU_0 = \frac{2}{2+0+1} = \frac{2}{3} \approx 0.6667$ .
Class 1: predicted at 3, true at 3, but only 2 coincide. One false positive (top middle) and one false negative (middle right) give $IoU_1 = \frac{2}{2+1+1} = 0.5$ .
Class 2: predicted at 4, true at 3; all 3 true pixels are hit plus one extra (middle right), so $IoU_2 = \frac{3}{3+1+0} = 0.75$ .
$mIoU = \frac{0.6667 + 0.5 + 0.75}{3} = \frac{1.9167}{3} \approx 0.6389$ , shown to 4 decimal places.
Notice that each mislabelled pixel costs two classes at once: middle right (true 1, predicted 2) is a false negative for class 1 and a false positive for class 2; likewise top middle (true 0, predicted 1) hurts classes 0 and 1.

Constraints

pred: Predicted segmentation map (H, W) with class indices
target: Ground truth segmentation (H, W)
num_classes: Total number of classes
Return: mIoU score (0-1)

Learn this in

Point Cloud Processing3D Computer Vision

More from CV: Recognition

Face Alignment with Landmarkshard Cosine Similarity Face Matchingmedium Top-K Accuracy Computationhard Optimal Verification Thresholdmedium Cosine Similarity for Face Verificationeasy

Concept primerPREVIEW

Background Knowledge

In semantic segmentation, each pixel in an image is assigned a class label (e.g., road, car, background). To evaluate how good these predictions are, we compare the predicted mask with the ground-truth mask per class. Intersection over Union (IoU) for a class $c$ measures how well the predicted region of that class overlaps the true region: it is the size of the intersection divided by the size of the union of predicted and true pixels for that class.

For class $c$ , define:

$P_c$ : set of pixels predicted as class $c$
$G_c$ : set of pixels whose ground-truth is class $c$

Then:

IoU_c = \frac{|P_c \cap G_c|}{|P_c \cup G_c|} = \frac{TP_c}{TP_c + FP_c + FN_c}

where $TP_c$ (true positives) are pixels correctly predicted as $c$ , $FP_c$ are pixels predicted as $c$ but actually another class, and $FN_c$ are pixels that are truly $c$ but predicted as something else. Mean IoU (mIoU) is simply the average of $IoU_c$ over all classes $C$ .

Algorithm/Approach

General pattern for this problem type:

Build a confusion matrix $M$ of size $C \times C$ , where $M[g, p]$ counts pixels with ground-truth class $g$ and predicted class $p$ .
For each class $c$ :
$TP_c = M[c, c]$
$FN_c =$ \sum_p M[c, p] - M[c, c] $(ground-truth$ c$, predicted something else)
$FP_c = \sum_g M[g, c] - M[c, c]$ (predicted $c$ , ground-truth something else)
Or more compactly:

IoU_c = \frac{TP_c}{TP_c + FP_c + FN_c}

Average valid class IoUs to get mIoU.

In practice, you compute the confusion matrix with a single vectorized operation, then derive all IoUs from it.

Step-by-Step Strategy

Assume you are given:

preds: an array of shape (H, W) (or (N, H, W)) with predicted class indices in [0, C-1]
gts: same shape with ground-truth classes in [0, C-1]
num_classes = C

Steps:

Flatten inputs

If batched, first reshape to 1D:

pred = preds.reshape(-1)
gt = gts.reshape(-1)

(Optional) Mask out ignored labels

If an ignore_index is used in the dataset, filter those pixels out before counting.

Build confusion matrix

Map each (gt, pred) pair to a single integer idx = gt * C + pred.
Use bincount (or similar) to count how many times each pair occurs.
Reshape back to (C, C):

conf = np.bincount(
gt * C + pred, minlength=C*C
).reshape(C, C)

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

Example

Constraints

Learn this in

More from CV: Recognition

Continue the full explanation

Editor locked