Many-to-One Output Selector

mediumPRORecurrent Neural Networks (RNNs)Deep LearningDeep Learning 1

Problem Statement

Select the final relevant output from a batch of variable-length sequences.

Background

In "Many-to-One" architectures (e.g., sentiment analysis), sequences in a batch often have different lengths. We pad shorter sequences but need to extract the hidden state at the last valid timestep for each sequence.

Your Task

Write a function get_final_states(hidden_states, sequence_lengths) that extracts the final hidden state for each sequence in the batch.

Input Format

hidden_states: numpy array of shape (batch_size, max_seq_len, hidden_dim)
sequence_lengths: list of integers, length of each sequence in the batch

Output Format

Return a numpy array of shape (batch_size, hidden_dim) containing the last valid hidden state for each sequence.

Example

Input

hidden_states shape (2, 3, 4), sequence_lengths = [2, 3]

Output

Shape (2, 4) with hidden_states[0,1,:] and hidden_states[1,2,:]

Reasoning

hidden_states is a $2 \times 3 \times 4$ block: 2 sequences, 3 timestep slots each, and a 4-number hidden vector in every slot. Filled with $0$ to $23$ in order, sequence 0 holds $[0,1,2,3]$ , $[4,5,6,7]$ , $[8,9,10,11]$ and sequence 1 holds $[12,\dots,15]$ , $[16,\dots,19]$ , $[20,\dots,23]$ .
sequence_lengths = [2, 3] says how many slots are real. Sequence 0 has only 2 real timesteps, so its third slot $[8,9,10,11]$ is padding.
A sequence of length $L$ occupies timesteps $0, 1, \dots, L-1$ (counting from zero), so its last real step is $t = L - 1$ : $t = 1$ for sequence 0 and $t = 2$ for sequence 1.
That picks $[4, 5, 6, 7]$ from sequence 0 and $[20, 21, 22, 23]$ from sequence 1.
The result is a $2 \times 4$ array: one row per sequence, one column per hidden unit. The time axis disappears because each sequence keeps exactly one timestep.
Notice that always taking the final slot would return the padding vector $[8,9,10,11]$ for sequence 0, which is exactly the mistake this selector avoids.

Constraints

1 <= batch_size <= 64
1 <= max_seq_len <= 512
1 <= hidden_dim <= 512
sequence_lengths[i] >= 1 for all i

Learn this in

Sequence ModelsNLP Study Plan

More from Deep Learning 1

Early Stopping Checkeasy L2 Regularization Penaltyeasy Inverted Dropout Forwardmedium Binary Cross-Entropy Losseasy Momentum Update Stepmedium

Concept primerPREVIEW

Many-to-One Output Selector: Comprehensive Background

1. Background Knowledge

Core Concepts

Many-to-One Architecture In sequence-to-sequence learning, many-to-one settings occur when multiple input sequences need to be reduced to a single output representation. This is fundamental in tasks like sentiment analysis, where an entire text sequence is classified into a single sentiment label.

Variable-Length Sequences and Padding Real-world sequences have different lengths. To process them efficiently in batches, shorter sequences are padded with dummy values (typically zeros) to match the longest sequence length. However, padding tokens don't carry meaningful information, so we must extract the hidden state at the last valid (non-padded) position for each sequence.

Hidden States in RNNs/LSTMs In recurrent architectures, the hidden state at each timestep encodes information about the sequence up to that point. The final hidden state (at the last valid timestep) serves as a compressed representation of the entire sequence, making it ideal for downstream classification or regression tasks.

Why This Matters

Without proper final state extraction, your model would use padding information, degrading performance. This is a critical preprocessing step in any sequence classification pipeline.

2. Algorithm Approach

Direct Indexing Strategy

The most efficient approach uses advanced indexing to directly access the correct position for each sequence:

Conceptual Steps:

Create a batch index array: [0, 1, 2,..., batch_size-1]
Create a position index array from sequence_lengths - 1 (convert to 0-indexed)
Use fancy indexing to extract: hidden_states[batch_indices, position_indices, :]

This avoids loops and leverages NumPy's vectorized operations.

Alternative: Loop-Based Approach

result = []
for i in range(batch_size):
 result.append(hidden_states[i, sequence_lengths[i] - 1, :])

While intuitive, this is slower for large batches.

3. Step-by-Step Strategy

Step 1: Understand the Indexing Problem

Shape: (batch_size, max_seq_len, hidden_dim)
For sequence i, the last valid position is sequence_lengths[i] - 1
Goal: Extract hidden_states[i, sequence_lengths[i] - 1, :] for all i

Step 2: Create Index Arrays

batch_indices = np.arange(batch_size) # [0, 1, 2,...]
position_indices = np.array(sequence_lengths) - 1 # Convert to 0-indexed

Step 3: Apply Fancy Indexing

NumPy's advanced indexing allows simultaneous indexing across multiple dimensions:

result = hidden_states[batch_indices, position_indices, :]

This selects hidden_states[0, seq_len-1, :], hidden_states[1, seq_len-1, :], etc.

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results