Softmax Cross-Entropy Loss

You are computing a single cross-entropy loss value from a predicted probability vector and the true class index. Mathematically, this is just taking the probability assigned to the correct class and applying $L = -\log(p_y)$, then rounding the result to 4 decimal places.

---

1. Background Knowledge

In multi-class classification (like image classification), a model outputs a vector of class scores (logits), which are converted to probabilities with the softmax function so that they are non-negative and sum to 1. This probability vector $p$ represents the model’s belief over all possible classes for a given input image.

The cross-entropy loss measures how well this predicted distribution $p$ matches the true label. For a single one-hot label (only one correct class), the cross-entropy simplifies to focusing only on the probability of the true class $p_y$: if the model assigns high probability to the correct class, the loss is small; if it assigns low probability, the loss is large. This makes cross-entropy a natural objective for training classification models with gradient-based methods.

---

2. Algorithm / Approach

The general pattern for this type of problem:

1. Select the predicted probability corresponding to the true class index $y$: $p_y = p[y]$.
2. Compute the negative log of that probability: $L = -\log(p_y)$.
3. Round the result to 4 decimal places for output.

Conceptually, you are just implementing the definition of cross-entropy for a single one-hot label.

---

3. Step-by-Step Strategy

1. Input parsing

• Read the vector of predicted probabilities p (e.g., as a list/array of floats).
• Read the integer y representing the true class index.

2. Access the correct probability

• Compute py = p[y].

3. Apply safety checks (optional but good practice)

• To avoid taking log(0), you can clamp:

eps = 1e-15
py = max(min(py, 1 - eps), eps)

4. Compute the loss

• Use natural logarithm:

import math
loss = -math.log(py)

5. Round to 4 decimal places

• Depending on language:

loss_rounded = round(loss, 4)

• Print or return loss_rounded.

---

4. Common Pitfalls

• Using the wrong index: Remember the label y is an index into p, not the probability itself.
• Log of zero: If p_y is exactly 0, log(0) is undefined and will cause errors or -inf. Clamping with a small epsilon avoids this in practice.
• Wrong logarithm base: Cross-entropy usually uses the natural log (ln). Using log10 or log2 changes the scale of the loss.
• Probabilities not summing to 1: In real models, the vector should come from a softmax and sum to 1. For this coding problem, you typically assume the input already satisfies this.

---

5. Time & Space Complexity

• Let $n$ be the number of classes (length of the probability vector).
• Time complexity:
• Accessing p[y] and computing a log is $O(1)$.
• Any parsing of the input vector is $O(n)$, but the core loss calculation is constant time.
• Space complexity:
• You store the probability vector $p$: $O(n)$.
• Additional variables (py, loss) use $O(1)$ extra space.