Embedding Cosine Similarity

easyFREEEmbeddings & Positional EncodingLLM 1: Foundations

Compute the cosine similarity between two embedding vectors.

Cosine similarity is defined as: $\text{cos\_sim}(a, b) = \frac{a \cdot b}{\|a\| \cdot \|b\|}$

Input:

a: a 1-D NumPy array of floats (vector a)
b: a 1-D NumPy array of floats (vector b), same length as a

Output: Return the cosine similarity as a float, rounded to 4 decimal places.

The example below shows each vector as space-separated floats, one per line.

Example

Input

1.0 0.0 0.0
0.0 1.0 0.0

Output

0.0

Reasoning

Input: two embedding vectors of the same length. Here $a = (1.0, 0.0, 0.0)$ points purely along the first axis and $b = (0.0, 1.0, 0.0)$ purely along the second.
Dot product: multiply matching components and add them: $a \cdot b = (1.0)(0.0) + (0.0)(1.0) + (0.0)(0.0) = 0.0$ . No component is non-zero in both vectors, so nothing survives.
Lengths: $\|a\| = \sqrt{1.0^2 + 0.0^2 + 0.0^2} = 1.0$ and likewise $\|b\| = 1.0$ . Both are already unit vectors.
Ratio: $\frac{0.0}{1.0 \times 1.0} = 0.0$ , rounded to 4 decimal places gives 0.0.
Notice: cosine similarity measures only the angle between the vectors, never their size. These two are at $90^\circ$ , so they are orthogonal: "unrelated" embeddings. Identical directions give $1$ (e.g. $a = b = (1, 2, 3)$ ), and opposite directions give $-1$ (e.g. $(1, 0)$ versus $(-1, 0)$ ). Scaling either vector by a positive number would not change the result.

Constraints

Vectors have the same dimension (1 <= D <= 100)
Vectors are not zero vectors
Output rounded to 4 decimal places

Learn this in

Retrieval-Augmented GenerationLLM Study Plan

More from LLM 1: Foundations

Transformer Block Forward Passhard Feed-Forward Networkeasy Layer Normalizationeasy KV-Cache Attentionhard Multi-Head Attentionmedium

Concept primer

Background Knowledge

The problem involves computing the cosine similarity between two vectors, which is a measure of similarity between them. The cosine similarity is defined as the dot product of the two vectors divided by the product of their magnitudes. This is often used in natural language processing and information retrieval to compare the semantic meaning of words or documents. The cosine similarity ranges from -1 (opposite direction) to 1 (same direction), with 0 indicating orthogonal vectors.

To understand the cosine similarity formula, we need to know about the dot product and vector magnitude. The dot product of two vectors $a = (a_1, a_2,..., a_n)$ and $b = (b_1, b_2,..., b_n)$ is given by $a \cdot b = a_1b_1 + a_2b_2 +... + a_nb_n$ . The magnitude of a vector $a$ is given by $\|a\| = \sqrt{a_1^2 + a_2^2 +... + a_n^2}$ . These concepts are fundamental in linear algebra and are used extensively in machine learning and data science.

In the context of embeddings, cosine similarity is used to compare the similarity between word or token embeddings. These embeddings are learned during the training process of a model and capture the semantic meaning of words in a high-dimensional space. By computing the cosine similarity between two embedding vectors, we can determine how similar the corresponding words or tokens are in terms of their meaning.

Algorithm/Approach

The general approach to solving this problem involves reading the input vectors, computing the dot product and magnitudes, and then applying the cosine similarity formula. This can be achieved using basic arithmetic operations and vector computations. The key is to ensure that the input vectors are properly parsed and that the computations are performed accurately.

Step-by-Step Strategy

To implement the solution, follow these steps:

Read the input vectors from the input lines and parse them into numerical arrays.
Compute the dot product of the two input vectors.
Compute the magnitudes of the two input vectors.
Apply the cosine similarity formula using the computed dot product and magnitudes.
Round the result to 4 decimal places and output it.

Common Pitfalls

When implementing the solution, watch out for the following:

Ensure that the input vectors are properly parsed and that the computations are performed accurately.
Be mindful of the data types used for the computations, as integer divisions or overflow may occur.
Make sure to round the result to 4 decimal places as required by the problem statement.

Time & Space Complexity

The time complexity of the solution is expected to be $O(n)$ , where $n$ is the dimensionality of the input vectors, since we need to iterate over the elements of the vectors to compute the dot product and magnitudes. The space complexity is expected to be $O(n)$ as well, since we need to store the input vectors and the intermediate results. However, the actual complexity may vary depending on the specific implementation and the programming language used.

solution.py

Python 3.13

Test results