Route to the Cheapest Model Meeting Quality

mediumPROProduction: Cost, Caching & EvalsAI AgentsAI Agents 1

Problem Statement

A router picks the cheapest model whose benchmark quality clears a required bar for a task. If none qualifies, fall back to the highest-quality model regardless of cost.

Background

Each model has a name, quality (0-1), and price (per call). Among models with quality >= min_quality, choose the lowest price (ties broken by higher quality, then name). If none meet the bar, return the model with the highest quality (ties: lower price, then name).

Your Task

def route_model(models, min_quality):

Return the chosen model's name.

Input Format

models: list of {"name": str, "quality": float, "price": float}, min_quality (float).

Output Format

A model name string.

Sample

m = [{"name":"big","quality":0.95,"price":10.0},{"name":"small","quality":0.8,"price":1.0}]
print(route_model(m, 0.75))

Output:

small

Example

Input

m = [{"name":"big","quality":0.95,"price":10.0},{"name":"small","quality":0.8,"price":1.0}]
print(route_model(m, 0.75))

Output

small

Reasoning

Filter for qualified models: We check each model against the minimum quality bar of $0.75$ .
- "big" has a quality of $0.95$ , which is $\ge 0.75$ , so it qualifies.
- "small" has a quality of $0.8$ , which is $\ge 0.75$ , so it also qualifies.
- The list of qualified models is ["big", "small"].
Select the cheapest option: Since there are qualified models, we choose the one with the lowest price to minimize cost.
- "big" has a price of $10.0$ .
- "small" has a price of $1.0$ .
- Comparing the prices, $1.0 < 10.0$ , so "small" is the cheaper option.
Verify tie-breakers: No tie-breaking is needed because the prices are distinct. If they were equal, we would look at quality (higher is better) and then name (alphabetical), but here the price difference is decisive.
Final Output: The name of the selected model is returned.
- The final output is small

Constraints

Prefer cheapest among quality >= min_quality (ties: higher quality, then name).
If none qualify, return the highest-quality model (ties: lower price, then name).
models is non-empty.

Learn this in

Workflow Patterns & LangGraph Stateful GraphsAgent Engineering Study Plan Deployment & OptimizationLLM Study Plan

More from AI Agents 1

Cheapest Cache TTL Meeting a Freshness SLAhard Bootstrap Confidence Interval Width for Eval Accuracymedium pass@k Estimatormedium Savings from a Semantic Cachemedium Cache Hit Rateeasy

Concept primerPREVIEW

1. Background Knowledge

This problem simulates a model router in a production AI system, where cost and quality are competing objectives. In real-world deployments, teams often maintain a fleet of models with different capability tiers and price points. The router’s job is to select the most cost-effective model that still meets a minimum quality threshold for a given task. This is a common pattern in cost-aware inference pipelines, where you want to avoid paying for a large, expensive model when a smaller one can handle the job adequately.

The core logic involves two distinct selection modes. In the primary mode, you filter for models that satisfy a quality constraint and then optimize for cost. In the fallback mode, triggered when no model meets the quality bar, you simply pick the best available model by quality, ignoring cost. Both modes require careful handling of tie-breaking rules, which are specified to ensure deterministic output when multiple models have identical scores.

Understanding lexicographic sorting is key here. When you need to pick a "best" model based on multiple criteria (e.g., lowest price, then highest quality, then alphabetical name), you are effectively defining a total order over the models. Python’s min and max functions accept a key argument that lets you define this ordering explicitly, making it straightforward to implement complex tie-breaking logic without writing manual comparison loops.

2. Algorithm Approach

The approach is a filter-then-optimize strategy with a fallback branch.

Filter: Create a subset of models where quality >= min_quality.
Primary Selection: If the subset is non-empty, find the model with the lowest price. If prices are tied, prefer the higher quality. If quality is also tied, prefer the lexicographically smaller name.
Fallback Selection: If the subset is empty, find the model with the highest quality from the full list. If qualities are tied, prefer the lower price. If prices are also tied, prefer the lexicographically smaller name.

This is not a dynamic programming or graph problem; it is a straightforward selection problem with conditional logic. The key insight is that the tie-breaking rules differ between the primary and fallback paths, so you cannot use a single sort key for both cases.

3. Step-by-Step Strategy

Define the primary key function: For the primary path, you want the "cheapest" model. In Python, min finds the smallest element. So, define a key that returns a tuple: (price, -quality, name). Using -quality ensures that higher quality comes first when prices are equal (since min picks the smallest tuple, and a more negative number is smaller). The name ensures alphabetical tie-breaking.
Filter the models: Use a list comprehension to create qualified = [m for m in models if m["quality"] >= min_quality].
Check if qualified is empty:

If not empty: Apply min(qualified, key=primary_key) and return the name of the result.
If empty: You need the fallback. Define a secondary key function for the fallback path. You want the "highest quality" model. Use max with a key that returns (quality, -price, name). Here, max picks the largest tuple. Higher quality is better. If quality is tied, you want the lower price, so use -price (since a more negative number is smaller, but max picks the largest, so you actually want the smallest price to be "best" in the tie-break. Wait, let's re-evaluate).

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results