RNN Single Step Forward

Problem Statement

Compute the hidden state for a single timestep in a standard RNN.

Background

The update rule for a vanilla RNN is:

$a_t = \tanh(W_{aa} \cdot a_{t-1} + W_{ax} \cdot x_t + b_a)$

Where:

• $a_t$ is the new hidden state
• $a_{t-1}$ is the previous hidden state
• $x_t$ is the input at timestep $t$
• $W_{aa}$ is the hidden-to-hidden weight matrix
• $W_{ax}$ is the input-to-hidden weight matrix
• $b_a$ is the bias vector

Your Task

Write a function rnn_step(prev_hidden, input_vec, W_aa, W_ax, b_a) that computes and returns the new hidden state using np.tanh.

Input Format

• prev_hidden: numpy array of shape (hidden_dim,)
• input_vec: numpy array of shape (input_dim,)
• W_aa: numpy array of shape (hidden_dim, hidden_dim)
• W_ax: numpy array of shape (hidden_dim, input_dim)
• b_a: numpy array of shape (hidden_dim,)

Output Format

Return a numpy array of shape (hidden_dim,) representing the new hidden state.