📘

Gradient Descent Step

Implement a single step of gradient descent for a 2D function, a fundamental optimization technique in machine learning. This process involves iteratively updating parameters to minimize a function.

Gradient descent is based on the concept of minimizing a function $f(x, y)$ by moving in the direction of the negative gradient, which is a vector of partial derivatives $\nabla f = [\partial f/\partial x, \partial f/\partial y]$ . The update rule for gradient descent can be broken down into steps:

Compute the gradient of the function at the current point.
Scale the gradient by the learning rate $\alpha$ .
Subtract the scaled gradient from the current point to obtain the new point.

\mathbf{x}_{new} = \mathbf{x}_{old} - \alpha \nabla f(\mathbf{x}_{old})

This technique is widely used in machine learning for training models.

Example:

Input:

f(x) = x²
f'(x) = 2x
x₀ = 5.0
learning_rate = 0.1
iterations = 50

Output:

x = 0.0 (minimum found)

Reasoning:

Gradient descent update rule:

$x_{\text{new}} = x_{\text{old}} - \eta \cdot \nabla f(x_{\text{old}})$

where $\eta$ is the learning rate.

Initial state: $x_0 = 5.0$
Iteration 1:
- Gradient: $f'(5.0) = 2 \times 5.0 = 10.0$
- Update: $x_1 = 5.0 - 0.1 \times 10.0 = 4.0$
Iteration 2:
- Gradient: $f'(4.0) = 2 \times 4.0 = 8.0$
- Update: $x_2 = 4.0 - 0.1 \times 8.0 = 3.2$
Pattern: Each step reduces $x$ by 20% $x \rightarrow x - 0.1 \times 2x = x \times (1 - 0.2) = 0.8x$
After many iterations: $x \rightarrow 0$ (the minimum of $x^2$ )
Result: $x \approx 0.0$

The function $f(x) = x^2$ has its minimum at $x = 0$ where $f'(x) = 0$ .

Constraints:

Starting point is [x, y]
Learning rate α > 0
Return new point [x', y'] rounded to 4 decimal places

Editor

Python 3.13.1

Test Results

0/0

Run code to see test results.

📘