Momentum Update Step

Problem Statement

Update velocity and parameters using Momentum optimization.

Background

Momentum helps accelerate gradients in the right direction and dampens oscillations. The update rules are:

$v = \beta \cdot v + (1 - \beta) \cdot dW$ $W = W - \alpha \cdot v$

Where:

• $v$ is the velocity (exponentially weighted average of gradients)
• $\beta$ is the momentum coefficient (typically 0.9)
• $\alpha$ is the learning rate
• $dW$ is the gradient

Your Task

Write a function momentum_step(weight, grad, velocity, beta, learning_rate) that returns a tuple of (updated_weight, updated_velocity).

Output Format

Return a tuple (weight, velocity) with values rounded to 4 decimal places.