TF-IDF Vectorizer

mediumPRORetrieval & SimilarityLLM 3: Applications & Evaluation

Implement TF-IDF vectorization for a corpus of documents.

TF (term frequency): tf(t, d) = count of t in d / total words in d IDF (inverse document frequency): idf(t) = log(N / df(t)) where N = total documents, df(t) = documents containing term t.

TF-IDF = tf × idf

Input:

Line 1: N (number of documents)
Next N lines: documents (space-separated words, lowercase)
Last line: query word

Output: TF-IDF score of the query word in each document, one per line, rounded to 4 decimal places.

Example

Input

3
the cat sat
the dog sat
the cat and dog
cat

Output

0.1352
0.0000
0.1352

Reasoning

We calculate the term frequency ( $tf$ $t f$ ) of the query word "cat" in each document:
- Document 1: $tf(cat, d_1) = \frac{1}{3}$ ,
- Document 2: $tf(cat, d_2) = 0$ ,
- Document 3: $tf(cat, d_3) = \frac{1}{5}$
We calculate the inverse document frequency ( $idf$ $i df$ ) of the query word "cat":
- $N = 3$ ,
- $df(cat) = 2$ ,
- $idf(cat) = \log(\frac{3}{2})$
We calculate the TF-IDF score for the query word "cat" in each document by multiplying $tf$ $t f$ and $idf$ $i df$ :
- Document 1: $TF-IDF = \frac{1}{3} \cdot \log(\frac{3}{2})$ ,
- Document 2: $TF-IDF = 0 \cdot \log(\frac{3}{2}) = 0$ ,
- Document 3: $TF-IDF = \frac{1}{5} \cdot \log(\frac{3}{2})$
The final output is the TF-IDF score of the query word in each document, rounded to 4 decimal places, resulting in the given sample output values.

Constraints

Use natural log (np.log)
If word not in document, TF-IDF = 0
If word not in any document, IDF = 0
Round to 4 decimal places

Learn this in

Text RepresentationNLP Study Plan

More from LLM 3: Applications & Evaluation

Beam Search Decoderhard INT8 Quantization Simulatormedium Nucleus (Top-P) Samplingmedium Top-K Samplingeasy Temperature Samplingeasy

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results