Contrastive Accuracy at the Diagonal

easyPROContrastive Objectives: CLIP and SigLIPVision-Language ModelsVLM 1: Vision-Language Models

Problem Statement

In a CLIP batch of N paired image-text examples, the correct match for image i is text i — the diagonal of the similarity matrix. Measure the image-to-text retrieval accuracy: the fraction of rows whose argmax lands on the diagonal.

Background

Given the N x N logits matrix S (row = image, column = text), the image-to-text prediction for row i is argmax_j S[i][j]. It is correct iff that argmax equals i. Break ties by the smallest column index (numpy's argmax default). Accuracy is the count of correct rows divided by N.

Your Task

Implement:

def contrastive_accuracy(logits):

Return the image-to-text accuracy as a float rounded to 4 decimals.

Input Format

logits: N x N nested list.

Output Format

A float rounded to 4 decimals.

Sample

print(contrastive_accuracy([[2.0, 1.0], [0.5, 3.0]]))

Output:

1.0

Example

Input

print(contrastive_accuracy([[2.0, 1.0], [0.5, 3.0]]))

Output

1.0

Reasoning

Convert the input nested list into a $2 \times 2$ matrix $S$ to represent the similarity scores between images and texts: $S = \begin{bmatrix} 2.0 & 1.0 \\ 0.5 & 3.0 \end{bmatrix}$
Determine the predicted text index for each image by finding the column index of the maximum value in each row (resolving ties by the smallest index, though none exist here):
- Row 0: $\max(2.0, 1.0) = 2.0$ at index $0$ .
- Row 1: $\max(0.5, 3.0) = 3.0$ at index $1$ .
- The prediction vector is $[0, 1]$ .
Compare these predictions against the ground truth diagonal indices $[0, 1]$ $[0, 1]$ to count correct matches:
- Image 0 predicted text 0 (Correct).
- Image 1 predicted text 1 (Correct).
- Total correct predictions = $2$ .
Calculate the accuracy by dividing the number of correct predictions by the total number of examples $N=2$ : $\text{Accuracy} = \frac{2}{2} = 1.0$
The final output is 1.0

Constraints

1 <= N <= 1000; the matrix is square.
Ties in a row go to the smallest column index.
Accuracy = correct_rows / N, rounded to 4 decimals.

Learn this in

Introduction to Vision-Language ModelsVLM Study Plan Vision EncodersVLM Study Plan Contrastive Learning & CLIPVLM Study Plan Training Recipes: Pretraining, SFT & RLHFVLM Study Plan

More from VLM 1: Vision-Language Models

Visual Token Pruning by Attention Masshard CHAIR-i and CHAIR-s Hallucination Ratesmedium Merge a LoRA Update into the Base Weightmedium QLoRA Memory Footprint Estimatemedium POPE Precision, Recall, and F1medium

Concept primerPREVIEW

1. Background Knowledge

Contrastive learning is a paradigm where a model learns by distinguishing between positive pairs (matching items) and negative pairs (non-matching items). In CLIP (Contrastive Language–Image Pre-training), each image is paired with its correct text caption, and the model learns to maximize the similarity between matched pairs while minimizing similarity between mismatched pairs.

The core computation produces an $N \times N$ logits matrix $S$ , where $S[i][j]$ represents the similarity score between image $i$ and text $j$ . The diagonal entries $S[i][i]$ correspond to the correct (positive) pairs. During training, a cross-entropy loss is applied row-wise: for each image $i$ , the model is trained so that $S[i][i]$ is the highest value in row $i$ relative to all other $S[i][j]$ values.

Retrieval accuracy is the standard evaluation metric. For image-to-text retrieval, you check whether the text with the highest similarity score for each image is indeed its correct caption. This is equivalent to checking whether the argmax of each row falls on the diagonal of the similarity matrix.

2. Algorithm Approach

This is a straightforward row-wise argmax problem. For each row $i$ in the $N \times N$ matrix:

Find the column index $j^* = \arg\max_j S[i][j]$ .
Check if $j^* = i$ (i.e., the argmax lands on the diagonal).
Count the number of rows where this condition holds.
Divide by $N$ to get the accuracy fraction.

The key insight is that you are performing an independent argmax operation on each row and comparing the result to the row index. No sorting or full matrix operations are needed beyond the per-row maximum search.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.