RNN Cell Forward
Implement a single step of a vanilla RNN cell.
The RNN update equations for one timestep:
ht=tanh(Whh⋅ht−1+Wxh⋅xt+bh)
Given the previous hidden state ht−1, current input xt, and weight matrices, compute the new hidden state ht.
Return ht as a list, rounded to 4 decimal places.
Example:
h_prev = [0, 0] x = [1] W_hh = [[0.1, 0.2], [0.3, 0.4]] W_xh = [[0.5], [0.6]] b_h = [0, 0]
[0.4621, 0.5370]
- First, we calculate the product of Whh and ht−1: Whh⋅ht−1=[0.10.30.20.4]⋅[00]=[00]
- Then, we calculate the product of Wxh and xt: Wxh⋅xt=[0.50.6]⋅[1]=[0.50.6]
- Next, we add the results of the previous steps and bh: [00]+[0.50.6]+[00]=[0.50.6]
- Finally, we apply the tanh function to get ht: ht=tanh[0.50.6]=[tanh(0.5)tanh(0.6)]=[0.46210.5370]
Constraints:
- h_prev: 1D list (hidden_size)
- x: 1D list (input_size)
- W_hh: 2D list (hidden_size x hidden_size)
- W_xh: 2D list (hidden_size x input_size)
- b_h: 1D list (hidden_size)
- Return 1D list of new hidden state values rounded to 4 decimal places
Background Knowledge
The problem involves implementing a single step of a vanilla RNN cell, which is a fundamental component of Recurrent Neural Networks (RNNs). RNNs are a type of neural network designed to handle sequential data, such as time series data or natural language processing tasks. The key characteristic of RNNs is their ability to maintain a hidden state, which allows them to capture temporal relationships in the data. The vanilla RNN cell is the simplest form of an RNN cell, and its update equations are based on a simple tanh activation function.
The update equation for the vanilla RNN cell is given by ht=tanh(Whh⋅ht−1+Wxh⋅xt+bh). This equation involves matrix multiplications between the weight matrices Whh and Wxh, the previous hidden state ht−1, and the current input xt. The result is then passed through a tanh activation function to produce the new hidden state ht. Understanding matrix multiplications, tanh activation functions, and the role of weight matrices and bias terms is essential for implementing this equation.
In the context of Machine Learning, RNNs are often used for tasks such as language modeling, text classification, and time series forecasting. The vanilla RNN cell is a basic building block for more complex RNN architectures, such as LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) cells. Understanding how to implement a single step of a vanilla RNN cell is crucial for building and working with more complex RNN models.
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.