L2-Normalize an Embedding

1. Background Knowledge

In Vision-Language Models (VLMs) like CLIP and SigLIP, images and text are mapped into a shared high-dimensional embedding space. The core training objective is contrastive learning, which pulls matching image-text pairs closer together and pushes non-matching pairs apart. To measure "closeness" in this space, models rely on cosine similarity, defined as the dot product of two vectors divided by the product of their magnitudes:

A key mathematical property is that if both vectors are L2-normalized (i.e., scaled to have a unit length of 1), the denominator becomes $1 \times 1 = 1$. Consequently, the cosine similarity simplifies to just the dot product: $\mathbf{a} \cdot \mathbf{b}$. This is computationally efficient and numerically stable, which is why normalization is a standard preprocessing step before computing similarities in contrastive objectives.

L2 normalization, also known as Euclidean normalization, projects a vector onto the unit sphere. For a vector $\mathbf{v} = [v_1, v_2, \dots, v_n]$, the L2 norm (or magnitude) is calculated as:

The normalized vector $\hat{\mathbf{v}}$ is then obtained by dividing each component by this norm. This operation preserves the direction of the vector while standardizing its length, ensuring that all embeddings are comparable regardless of their original scale.

2. Algorithm Approach

The solution follows a straightforward vector arithmetic pattern:

1. Compute the Norm: Iterate through the input vector to calculate the sum of squares of all elements, then take the square root to get the L2 norm.
2. Handle Edge Cases: Check if the computed norm is zero. If so, return the original vector (or a list of zeros) to avoid division by zero.
3. Scale the Vector: Divide each element of the input vector by the computed norm.
4. Format Output: Round each resulting element to 4 decimal places as specified.

This is a linear-time operation that does not require complex data structures or iterative refinement.

3. Step-by-Step Strategy

1. Initialize Sum of Squares: Create a variable (e.g., sum_sq) initialized to 0.
2. Calculate Norm:

• Loop through each element x in the input list v.
• Add x * x to sum_sq.
• After the loop, compute norm = sqrt(sum_sq).

3. Zero-Vector Guard:

• If norm == 0, return [0.0] * len(v) (or the original list, depending on strict interpretation, but usually zeros are expected for a zero vector).

4. Normalize:

• Create a new list where each element x is replaced by x / norm.

5. Round and Return:

• Apply round(value, 4) to each element in the normalized list.
• Return the final list.

Example logic snippet:

import math

def l2_normalize(v):
 sum_sq = sum(x * x for x in v)
 norm = math.sqrt(sum_sq)
 if norm == 0:
 return [0.0] * len(v)
 return [round(x / norm, 4) for x in v]

4. Common Pitfalls

• Division by Zero: Forgetting to check if the norm is zero will cause a ZeroDivisionError when the input is [0, 0, 0]. Always guard against this.
• Integer Division: In Python 3, / performs float division, which is correct. However, if you accidentally use // (floor division), you will lose precision and get incorrect results.
• Rounding Precision: The problem specifically asks for rounding to 4 decimals. Using round(x, 2) or returning full precision will fail test cases. Ensure you apply round() to each element individually.
• Modifying the Input: Avoid modifying the original list v in place. Create a new list for the normalized values to maintain function purity.
• Floating-Point Errors: When checking if the norm is zero, direct comparison norm == 0 is generally safe for this specific problem since the input is a list of numbers and the sum of squares is exactly 0 only if all inputs are 0. However, in more complex numerical contexts, using a small epsilon (e.g., 1e-8) is a safer practice.

5. Time & Space Complexity

• Time Complexity: $O(n)$, where $n$ is the length of the input vector. We iterate through the vector once to compute the sum of squares and once more to normalize and round the elements.
• Space Complexity: $O(n)$, because we store the normalized vector in a new list of the same size as the input. The input list itself is not modified.