Effective Horizon of a Discount Factor

mediumPROMarkov Decision Processes and Bellman EquationsReinforcement LearningRL 1: Foundations

Problem Statement

The effective planning horizon implied by a discount factor is often summarized as 1 / (1 - gamma) — the sum of the geometric discount series sum_{t=0}^inf gamma^t. For gamma == 1 the horizon is infinite; represent that with float('inf').

Implement effective_horizon(gamma) returning a float.

Example

Input

effective_horizon(0.9)

Output

10.0

Reasoning

Check if the discount factor $\gamma = 0.9$ is greater than or equal to $1.0$ to determine if the horizon is infinite; since $0.9 < 1.0$ , the horizon is finite.
Calculate the denominator of the geometric series sum by subtracting the discount factor from $1$ : $1.0 - 0.9 = 0.1$ .
Compute the effective horizon by dividing $1$ by this denominator: $1.0 / 0.1 = 10.0$ .
The final output is 10.0

Constraints

0.0 <= gamma <= 1.0
gamma == 1.0 returns float('inf').
Otherwise return 1 / (1 - gamma).

Learn this in

Introduction to Reinforcement LearningReinforcement Learning Markov Decision ProcessesReinforcement Learning

More from RL 1: Foundations

GAE Over a Full Trajectoryhard PPO Clipped Objective (Single Sample)medium TD Residuals for GAEmedium Return Standardization Baselinemedium REINFORCE Loss Termmedium

Concept primerPREVIEW

1. Background Knowledge

In Markov Decision Processes (MDPs), the discount factor $\gamma \in [0, 1]$ controls how much an agent values future rewards relative to immediate ones. The total expected return from time $t=0$ onward is defined as:

G_t = \sum_{k=0}^{\infty} \gamma^k R_{t+k}

The term $\gamma^k$ acts as a weight that decays exponentially. When $\gamma$ is close to 1, the agent is "patient" and considers distant rewards almost as important as near-term ones. When $\gamma$ is small, the agent is "myopic" and mostly cares about immediate outcomes.

The effective horizon is a heuristic measure of how many steps into the future the agent effectively "sees." It is derived from the sum of the pure discount weights (ignoring actual reward magnitudes):

H = \sum_{k=0}^{\infty} \gamma^k

This is a geometric series. For $|\gamma| < 1$ , the series converges to a finite value. For $\gamma = 1$ , every term equals 1, so the sum diverges to infinity. This distinction is critical in reinforcement learning: a finite horizon implies the agent's planning is bounded, while an infinite horizon implies the agent must consider all future consequences equally.

2. Algorithm Approach

This problem is a direct application of the closed-form solution for a geometric series. Rather than summing terms iteratively (which would be inefficient and prone to floating-point accumulation errors), you should use the analytical formula:

\sum_{k=0}^{\infty} r^k = \frac{1}{1 - r} \quad \text{for } |r| < 1

The algorithmic pattern here is:

Check for the special case where the series diverges ( $\gamma = 1$ ).
Apply the closed-form formula for all other valid inputs.

This is a constant-time mathematical evaluation, not an iterative computation.

3. Step-by-Step Strategy

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results