Recall@K for Image-Text Retrieval

Problem Statement

Given a query-by-gallery similarity matrix and the index of the correct gallery item for each query, compute Recall@K for several values of K.

Background

Recall@K is the standard retrieval metric for CLIP-style models on COCO and Flickr30k: the fraction of queries whose ground-truth match appears anywhere in the top K results.

For query i, sort the gallery by descending similarity and find the 0-indexed position of ground_truth[i]:

$\text{rank}_i = \left|\{\, j : \text{sim}[i][j] > \text{sim}[i][g_i] \,\}\right| \quad\text{(with ties broken by index)}$

$\text{R@}K = \frac{1}{n}\sum_{i=1}^{n} \mathbb{1}\!\left[\text{rank}_i < K\right]$

Two details decide whether your number matches a published one. First, ranks are 0-indexed against K, so R@1 means rank == 0. Second, ties must be broken deterministically - here, toward the lower gallery index - otherwise a model that returns identical scores appears to score differently on every run.

Recall@K is monotone non-decreasing in K, and R@m over an m-item gallery is always 1.0.

Your Task

Implement:

def recall_at_k(sim_matrix, ground_truth, ks):

Return a list of floats, one per entry of ks in the given order, each rounded to 4 decimals.

Input Format

• sim_matrix - n x m nested list of floats, row i scoring query i against every gallery item
• ground_truth - list of n ints, the correct gallery index per query
• ks - list of positive ints (not necessarily sorted)

Output Format

A list of floats rounded to 4 decimals, same length and order as ks.

Sample

sims = [[0.9, 0.2, 0.1], [0.3, 0.4, 0.8], [0.5, 0.7, 0.6]]
print(recall_at_k(sims, [0, 2, 0], [1, 2]))

Output:

[0.6667, 0.6667]