📘

Temperature Sampler

EasyText Generation & Advanced 📚 NLP 3: Advanced

Given a list of logits (raw scores) for vocabulary words and a temperature value, apply temperature scaling and compute the resulting probability distribution using softmax.

Temperature scaling divides each logit by the temperature before applying softmax: $p_i = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}$

Input format:

Line 1: Comma-separated word:logit pairs (e.g., "cat:2.0,dog:1.0,fish:0.5")
Line 2: Temperature T (float)

Output: Each word and its probability (rounded to 4 decimal places), sorted by probability descending, one per line as "word probability".

Example:

Input:

cat:2.0,dog:1.0,fish:0.5
1.0

Output:

cat 0.6285
dog 0.2312
fish 0.1402

Reasoning:

Step 1: Divide logits by temperature (T=1.0) cat: 2.0/1.0 = 2.0, dog: 1.0/1.0 = 1.0, fish: 0.5/1.0 = 0.5

Step 2: Apply softmax e^2.0 ≈ 7.389, e^1.0 ≈ 2.718, e^0.5 ≈ 1.649 Sum ≈ 11.756

Step 3: Normalize cat: 7.389/11.756 ≈ 0.6285 dog: 2.718/11.756 ≈ 0.2312 fish: 1.649/11.756 ≈ 0.1402

Constraints:

Temperature T > 0
Apply softmax after dividing logits by T
Round probabilities to 4 decimal places
Output sorted by probability descending (ties: alphabetical)

More from NLP 3: Advanced

ROUGE-1 Score CalculatorEasy Context Window FinderEasy Keyword Answer ExtractorEasy Levenshtein DistanceEasy Word-Level TranslatorEasy

Background Knowledge

The problem involves applying temperature scaling and softmax to a list of logits. Logits are raw, unnormalized scores that a model outputs for each class or word in a vocabulary. Softmax is a function that takes these logits and converts them into a probability distribution, where each word has a probability between 0 and 1, and the probabilities sum up to 1. Temperature scaling is a technique used to control the "softness" of the softmax output. By dividing each logit by a temperature value, we can adjust the level of uncertainty in the output probabilities.

The softmax function is defined as $p_i = \frac{e^{z_i}}{\sum_j e^{z_j}}$ , where $z_i$ is the logit for the $i^{th}$ word, and $p_i$ is the corresponding probability. When we apply temperature scaling, we replace $z_i$ with $z_i / T$ , where $T$ is the temperature. This has the effect of "flattening" the probability distribution when $T$ is high, and "sharpening" it when $T$ is low. Understanding how softmax and temperature scaling work is crucial to solving this problem.

In the context of Natural Language Processing (NLP), temperature scaling can be used to control the level of randomness in text generation models. For example, a high temperature can result in more diverse and unpredictable text, while a low temperature can result in more repetitive and predictable text. The Temperature Sampler problem requires applying these concepts to a given list of logits and temperature value, and computing the resulting probability distribution.

Algorithm/Approach

The general approach to solving this problem involves the following steps:

Parse the input data, including the word-logit pairs and the temperature value
Apply temperature scaling to the logits
Compute the softmax of the scaled logits
Sort the resulting probabilities in descending order
Output each word and its corresponding probability

This approach involves a combination of data parsing, mathematical computations, and sorting.

Step-by-Step Strategy

To implement the solution, follow these steps:

Parse the input string into a dictionary or list of word-logit pairs
Extract the temperature value from the input
Apply temperature scaling to each logit
Compute the softmax of the scaled logits using the formula: $p_i = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}$
Sort the resulting probabilities in descending order
Output each word and its corresponding probability, rounded to 4 decimal places

Common Pitfalls

When implementing the solution, watch out for the following:

Make sure to handle the input data correctly, including parsing the word-logit pairs and extracting the temperature value
Be careful when applying temperature scaling, as dividing by zero can result in errors
Use a stable implementation of the softmax function to avoid numerical issues
Make sure to sort the probabilities in descending order, as required by the problem statement

Time & Space Complexity

The time complexity of the solution is expected to be O(n log n) due to the sorting step, where n is the number of word-logit pairs. The space complexity is expected to be O(n) for storing the word-logit pairs and the resulting probabilities. However, the exact complexity may depend on the specific implementation and the size of the input data.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.