Horizontal Pod Autoscaler Replica Count

1. Background Knowledge

The Horizontal Pod Autoscaler (HPA) is a core Kubernetes controller that automatically scales the number of pod replicas in a deployment, replication controller, or replica set based on observed metrics such as CPU utilization or custom application metrics. The primary goal is to maintain system stability and performance by ensuring that resources match demand. In machine learning systems, this is particularly critical for inference services where latency and throughput must remain within strict Service Level Objectives (SLOs).

The control logic relies on a feedback loop that compares the current metric (e.g., average CPU usage per pod) against a target metric (the desired usage level). If the current usage exceeds the target, the system scales up; if it is below, it scales down. However, naive scaling can lead to oscillation, where the system rapidly flips between scaling up and down due to minor fluctuations around the target. To prevent this, HPAs implement a tolerance band (often 10%) where no action is taken if the metric is close enough to the target.

Additionally, scaling decisions are constrained by clamping bounds (minReplicas and maxReplicas) to prevent resource exhaustion or under-provisioning. The calculation involves a ceiling function because fractional pods are not possible, and under-provisioning (rounding down) is generally more costly than over-provisioning (rounding up) in terms of latency spikes.

2. Algorithm Approach

The problem requires implementing a deterministic control loop based on a specific mathematical formula. The approach is direct computation with conditional logic for stability and bounds checking.

1. Calculate Ratio: Determine the ratio of current_metric to target_metric.
2. Check Tolerance: Evaluate if the ratio is within the acceptable tolerance band (typically $\pm 10\%$ of 1.0). If it is, return the current_replicas unchanged to prevent flapping.
3. Compute Desired Replicas: If outside the tolerance, apply the HPA formula: $\text{desired} = \lceil \text{current\_replicas} \times \text{ratio} \rceil$.
4. Clamp Result: Ensure the final result lies within the [min_replicas, max_replicas] range.

This approach avoids complex iterative algorithms or simulations, relying instead on precise arithmetic and boundary checks.

3. Step-by-Step Strategy

1. Handle Edge Cases: Although the problem implies valid inputs, ensure target_metric is not zero to avoid division errors. If target_metric is 0, the behavior is undefined in standard HPA, but for this problem, assume valid positive inputs.
2. Compute the Ratio: Calculate ratio = current_metric / target_metric.
3. Apply Tolerance Check:

• The tolerance condition is abs(ratio - 1.0) <= 0.1.
• If this condition is True, the system is stable. Return current_replicas.

4. Calculate Raw Desired Replicas:

• Use the formula: raw_desired = current_replicas * ratio.
• Apply the ceiling function: desired = ceil(raw_desired). In Python, use math.ceil().

5. Clamp the Value:

• Ensure desired is at least min_replicas.
• Ensure desired is at most max_replicas.
• This can be done using max(min_replicas, min(max_replicas, desired)).

6. Return Result: Cast the final result to an integer and return it.

4. Common Pitfalls

• Floating Point Precision: When checking abs(ratio - 1.0) <= 0.1, be aware of floating-point inaccuracies. While standard float precision is usually sufficient for this problem, always use math.ceil rather than integer casting (int()) for the final calculation, as (int()) truncates towards zero (floor for positive numbers), which violates the "under-provisioning is expensive" rule.
• Tolerance Logic Error: A common mistake is checking if current_metric is within 10% of target_metric directly. The problem specifies the ratio must be within 10% of 1.0. These are mathematically equivalent, but implementing the ratio check explicitly reduces confusion.
• Clamping Order: Ensure clamping happens after the ceiling operation. Clamping before ceiling could result in a value that, when ceiled, exceeds max_replicas.
• Integer Division: In Python 3, / performs float division. Ensure you are not accidentally using integer division // for the ratio calculation, which would yield incorrect results for non-integer ratios.
• Missing Imports: Remember to import math for math.ceil.

5. Time & Space Complexity

• Time Complexity: $O(1)$. The solution involves a constant number of arithmetic operations (division, multiplication, absolute value, ceiling) and comparisons. It does not depend on the size of any input data structure, only on the numerical values provided.
• Space Complexity: $O(1)$. The algorithm uses a fixed amount of extra space for variables (ratio, desired, etc.), regardless of input size. No additional data structures are allocated.