Tail Latency Percentiles (Nearest-Rank)

Problem Statement

Mean latency is the least useful number on an inference dashboard: it hides the tail, and the tail is what users feel and what your SLO is written against. Compute the tail percentiles of a batch of request latencies.

Background

There are several incompatible percentile definitions, and a dashboard that mixes them will disagree with itself. This problem uses nearest-rank, with no interpolation:

rank    = ceil(p / 100 * n)      # 1-based, clamped into [1, n]
P_p     = sorted_samples[rank - 1]

Nearest-rank always returns a value that was actually observed. With 100 samples of 1..100, p99 is exactly 99 — an interpolating definition would report 99.01, a latency no request ever had.

Because p95 picks the ceil(0.95 * n)-th smallest sample, a handful of slow requests move it a long way. That is the point: it is a tail statistic.

Your Task

Implement:

def latency_percentiles(samples):

Return a dict with keys "count", "p50", "p95", "p99", "max", in that order. "count" is an int; the rest are the selected samples rounded to 2 decimal places.

Input Format

• samples: list of latencies in milliseconds (ints or floats), in arbitrary order.

Output Format

• The dict described above.

Sample

print(latency_percentiles([12, 15, 11, 480, 13, 14, 12, 16, 13, 12]))

Output:

{'count': 10, 'p50': 13, 'p95': 480, 'p99': 480, 'max': 480}

The median sits at 13 ms while a single 480 ms outlier owns both tail percentiles — a service whose p50 looks healthy and whose p95 is 37x worse.