Spot vs On-Demand Break-Even

mediumPROPipelines, Reliability & CostSystems & InfrastructureML Systems & Infrastructure 1

Problem Statement

GPU spot instances are 60-90% cheaper than on-demand, but AWS can reclaim them with two minutes' notice. Each reclaim costs you the time to relaunch and restore from the last checkpoint. Work out whether spot is actually cheaper for a given training job, and at what spot price the two break even.

Background

The naive comparison — spot price versus on-demand price — is wrong, because interruptions make the job take longer, and you pay for that extra time too.

Model:

expected_interruptions = hours * interruption_prob      # per-instance-hour reclaim probability
effective_hours        = hours + expected_interruptions * restart_overhead_hours
on_demand_cost         = hours * instances * on_demand_price
spot_cost              = effective_hours * instances * spot_price
savings                = on_demand_cost - spot_cost
savings_pct            = 100 * savings / on_demand_cost

The break-even spot price is the price per hour at which spot costs exactly the same as on-demand:

break_even_spot_price = on_demand_price * hours / effective_hours

Below it spot wins, above it the restart overhead eats the discount. Note instances cancels out — it scales both sides equally.

Frequent interruptions with an expensive restart (a big model with slow checkpoint restore) can push savings negative even at a headline 70% discount. That is the answer this calculation is for.

Your Task

Implement:

def spot_vs_on_demand(hours, instances, on_demand_price, spot_price, interruption_prob, restart_overhead_hours):

Return a dict with keys "effective_hours", "on_demand_cost", "spot_cost", "savings", "savings_pct", "break_even_spot_price", in that order. Round the first five to 2 decimals and "break_even_spot_price" to 4.

Input Format

hours: uninterrupted wall-clock hours the job would need.
instances: number of instances.
on_demand_price, spot_price: USD per instance-hour.
interruption_prob: probability that an instance is reclaimed in any given hour.
restart_overhead_hours: hours lost per interruption (relaunch plus checkpoint restore).

Output Format

The dict described above.

Sample

print(spot_vs_on_demand(48, 4, 3.06, 0.90, 0.02, 0.25))

Output:

{'effective_hours': 48.24, 'on_demand_cost': 587.52, 'spot_cost': 173.66, 'savings': 413.86, 'savings_pct': 70.44, 'break_even_spot_price': 3.0448}

At 2% hourly reclaim risk you expect 0.96 interruptions over 48 hours, adding about 14 minutes of restart time. Spot still wins by a mile — you would only lose out above $3.0448/hr.

Example

Input

print(spot_vs_on_demand(48, 4, 3.06, 0.90, 0.02, 0.25))

Output

{'effective_hours': 48.24, 'on_demand_cost': 587.52, 'spot_cost': 173.66, 'savings': 413.86, 'savings_pct': 70.44, 'break_even_spot_price': 3.0448}

Reasoning

48 hours at a 2% hourly reclaim rate gives 0.96 expected interruptions, each costing 0.25 h, so the job really occupies 48.24 instance-hours. On-demand is 48 * 4 * 3.06 = 587.52 while spot is 48.24 * 4 * 0.90 = 173.66, a 70.44% saving. Spot only stops paying once its price reaches 3.06 * 48 / 48.24 = 3.0448 per hour.

Constraints

hours > 0, instances >= 1, prices >= 0
0 <= interruption_prob <= 1, restart_overhead_hours >= 0
Expected interruptions is hours * interruption_prob (no rounding to whole interruptions)
savings and savings_pct may be negative
Round effective_hours, on_demand_cost, spot_cost, savings, savings_pct to 2 decimals and break_even_spot_price to 4

Learn this in

Cloud & AWS EssentialsSystems & Networking Orchestration & ScalingSystems & Networking

More from ML Systems & Infrastructure 1

Spot vs On-Demand Expected-Cost Decisionmedium Minimum-Cost Job Scheduling Across Priced Windowshard Deduplicate Retried Pipeline Runs by Idempotency Keymedium Retry with Capped Exponential Backoff and Jitter Capmedium Critical Path Length of a Pipeline DAGmedium

Concept primerPREVIEW

1. Background Knowledge

In cloud computing, particularly for machine learning workloads, cost optimization is a critical engineering challenge. On-Demand instances provide guaranteed availability at a premium price, while Spot instances offer significant discounts (often 60-90%) but carry the risk of preemption. Cloud providers can reclaim Spot instances with short notice (e.g., two minutes) when capacity is needed elsewhere. This introduces interruption overhead, which includes the time to relaunch the instance, download model weights, and restore the training state from the last checkpoint.

The core concept here is Expected Value. You cannot simply compare the hourly rates because the effective duration of a Spot job is longer than its nominal duration. The effective hours account for the base training time plus the cumulative time lost to interruptions. If interruptions are frequent or the restart process is slow (large models, slow storage), the extra time spent on Spot instances can negate the hourly discount, making On-Demand cheaper despite the higher rate.

This problem models a break-even analysis. It calculates the point where the total cost of using Spot instances (including overhead) equals the total cost of using On-Demand instances. Understanding this helps engineers decide whether the risk of preemption is worth the potential savings for a specific job configuration.

2. Algorithm Approach

The approach is a direct mathematical simulation based on the provided formulas. It involves calculating derived metrics step-by-step to ensure accuracy and proper rounding.

Calculate Expected Interruptions: Multiply the total job hours by the hourly interruption probability. This gives the average number of times the job will be interrupted.
Calculate Effective Hours: Add the base hours to the total overhead time caused by expected interruptions.
Calculate Costs: Compute the total cost for both On-Demand and Spot scenarios using their respective prices, instance counts, and durations (base for On-Demand, effective for Spot).
Derive Savings: Subtract Spot cost from On-Demand cost to find absolute savings, then calculate the percentage savings relative to the On-Demand cost.
Calculate Break-Even Price: Determine the Spot price at which the total Spot cost equals the On-Demand cost. This is derived by dividing the On-Demand cost by the total Spot instance-hours (effective hours * instances).

3. Step-by-Step Strategy

Compute expected_interruptions: Multiply hours by interruption_prob. This represents the average number of reclaims.

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results