Token Accounting Across a Multi-Model Agent Run

Problem Statement

The OpenTelemetry GenAI conventions exist so that every span an agent emits carries the same usage attributes - input tokens, output tokens, cached input tokens - regardless of which provider served the call. Once you have that, the cost of a run is arithmetic. Getting the arithmetic wrong by a factor of ten is the standard way to be surprised by a bill.

Background

Providers quote prices per million tokens. A usage record looks like:

{"model": "big", "input_tokens": 1000, "cached_input_tokens": 800, "output_tokens": 200}

The trap: input_tokens is the total input, and cached_input_tokens is the subset of it that was served from the prompt cache. The cached portion is billed at the (much lower) cached rate, and only the remainder at the full input rate:

$\text{cost} = \frac{(\text{input} - \text{cached}) \cdot p_{in} + \text{cached} \cdot p_{cached} + \text{output} \cdot p_{out}}{10^6}$

Double-counting the cached tokens - charging them at both rates - is the single most common bug in agent cost dashboards.

Your Task

Implement:

def compute_cost(usage, prices):

• usage: list of usage records as above; cached_input_tokens may be absent, meaning 0.
• prices: dict of model -> {"input": float, "cached_input": float, "output": float}, all per million tokens.

Return:

• total_cost: total across every record, rounded to 6 decimal places
• by_model: dict of model -> that model's total cost, rounded to 6 dp, built by iterating the model names in sorted order
• total_tokens: sum(input_tokens + output_tokens) - cached tokens are already inside input_tokens, so do not add them again

Input/Output Format

Returns the three-key dict. All money values are rounded to 6 dp; total_tokens is an int.

Sample

usage = [{"model": "big", "input_tokens": 1000, "cached_input_tokens": 800, "output_tokens": 200}]
prices = {"big": {"input": 3.0, "cached_input": 0.3, "output": 15.0}}
print(compute_cost(usage, prices)["total_cost"])
# 0.00384

200 fresh input tokens at $3/M = $0.0006, 800 cached at $0.30/M = $0.00024, 200 output at $15/M = $0.003. Total $0.00384.