Binary Cross-Entropy Loss

easyFREEClassificationMachine Learning 1

Compute the binary cross-entropy loss (log loss) for a set of predictions.

Given true labels $y_i \in \{0, 1\}$ and predicted probabilities $\hat{y}_i \in (0, 1)$ :

$BCE = -\frac{1}{n}\sum_{i=1}^{n}[y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)]$

To avoid $\log(0)$ , clip predictions to the range $[\epsilon, 1-\epsilon]$ where $\epsilon = 10^{-7}$ .

Return the loss rounded to 4 decimal places.

Example

Input

y_true = [1, 0, 1]
y_pred = [0.9, 0.1, 0.8]

Output

0.1446

Reasoning

y_true holds the actual class of each of the $n = 3$ samples ( $1$ = positive, $0$ = negative); y_pred holds the model's probability that each sample is positive.
All three predictions already lie well inside $[10^{-7}, 1 - 10^{-7}]$ , so the clipping leaves them unchanged here; it only matters for predictions at (or within $10^{-7}$ of) $0$ or $1$ .
Each sample contributes only one of the two terms: when $y_i = 1$ the loss is $-\ln(\hat{y}_i)$ , when $y_i = 0$ it is $-\ln(1 - \hat{y}_i)$ , i.e. minus the natural log ( $\log$ in the formula) of the probability given to the true class.
Sample 1: $-\ln(0.9) \approx 0.1054$ . Sample 2: true class is $0$ , probability given to it is $1 - 0.1 = 0.9$ , so again $\approx 0.1054$ . Sample 3: $-\ln(0.8) \approx 0.2231$ .
Average: $\frac{0.1054 + 0.1054 + 0.2231}{3} \approx 0.1446$ , the output 0.1446 (rounded to 4 decimal places).
Notice the least confident correct prediction (sample 3) contributes the most; a confident wrong prediction would make the loss blow up.

Constraints

y_true: list of 0s and 1s
y_pred: list of predicted probabilities (0 to 1)
Clip predictions to [1e-7, 1-1e-7] before computing log
Return a single float rounded to 4 decimal places

Learn this in

Mathematical FoundationsFoundations ClassificationML Study Plan Model EvaluationML Study Plan

More from Machine Learning 1

RBF Kernel Matrixhard Polynomial Kernelmedium Hinge Losseasy Linear Decision Boundaryeasy Gradient Boosting Residual Stephard

Concept primer

Background Knowledge

The binary cross-entropy loss is a fundamental concept in machine learning, particularly in classification problems. It measures the difference between the predicted probabilities and the true labels. The goal is to minimize this loss function to achieve better predictions. The binary cross-entropy loss is defined as $BCE = -\frac{1}{n}\sum_{i=1}^{n}[y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)]$ , where $y_i$ represents the true labels and $\hat{y}_i$ represents the predicted probabilities.

In machine learning, loss functions are used to evaluate the performance of a model. The binary cross-entropy loss is commonly used in binary classification problems, where the target variable is binary (0 or 1). This loss function is also known as log loss. To avoid $\log(0)$ , which is undefined, predictions are clipped to the range $[\epsilon, 1-\epsilon]$ , where $\epsilon$ is a small value, typically $10^{-7}$ .

The binary cross-entropy loss is a differentiable function, which makes it suitable for optimization using gradient descent. The gradient descent algorithm iteratively updates the model's parameters to minimize the loss function. Understanding the binary cross-entropy loss and its properties is essential for building and optimizing machine learning models, especially in classification tasks.

Algorithm/Approach

The general approach to solving this problem involves implementing the binary cross-entropy loss formula. This requires iterating over the true labels and predicted probabilities, computing the loss for each sample, and summing up the losses. To avoid $\log(0)$ , the predictions need to be clipped to the range $[\epsilon, 1-\epsilon]$ . The final loss value is then rounded to 4 decimal places.

Step-by-Step Strategy

To implement the solution, follow these steps:

Clip the predicted probabilities to the range $[\epsilon, 1-\epsilon]$ to avoid $\log(0)$ .
Iterate over the true labels and clipped predicted probabilities.
For each sample, compute the loss using the binary cross-entropy loss formula: $y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)$ .
Sum up the losses for all samples and divide by the total number of samples.
Round the final loss value to 4 decimal places.

Common Pitfalls

When implementing the solution, watch out for the following:

Make sure to clip the predicted probabilities to avoid $\log(0)$ .
Use the correct binary cross-entropy loss formula.
Iterate over the true labels and predicted probabilities correctly.
Round the final loss value to 4 decimal places as required.

Time & Space Complexity

The time complexity of the solution is O(n), where n is the number of samples, since we need to iterate over the true labels and predicted probabilities. The space complexity is O(1), since we only need to store the final loss value and a few intermediate variables. Note that the space complexity assumes that the input arrays (true labels and predicted probabilities) are already allocated and do not need to be stored in addition to the output.

solution.py

Python 3.13

Test results