📘

Binary Cross-Entropy Loss

EasyClassification 📚 Machine Learning 1

Compute the binary cross-entropy loss (log loss) for a set of predictions.

Given true labels $y_i \in \{0, 1\}$ and predicted probabilities $\hat{y}_i \in (0, 1)$ :

$BCE = -\frac{1}{n}\sum_{i=1}^{n}[y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)]$

To avoid $\log(0)$ , clip predictions to the range $[\epsilon, 1-\epsilon]$ where $\epsilon = 10^{-7}$ .

Return the loss rounded to 4 decimal places.

Example:

Input:

y_true = [1, 0, 1]
y_pred = [0.9, 0.1, 0.8]

Output:

0.1446

Reasoning:

First, we clip the predicted probabilities to the range $[\epsilon, 1-\epsilon]$ where $\epsilon = 10^{-7}$ : $y_{pred} = [0.9, 0.1, 0.8]$ remains the same since all values are within the range.
Then, we calculate the binary cross-entropy loss for each sample:
- For $y_i = 1$ and $\hat{y}_i = 0.9$ , the loss is $-\log(0.9)$
- For $y_i = 0$ and $\hat{y}_i = 0.1$ , the loss is $-\log(1-0.1) = -\log(0.9)$
- For $y_i = 1$ and $\hat{y}_i = 0.8$ , the loss is $-\log(0.8)$
Next, we calculate the total loss by summing the individual losses and dividing by the number of samples $n=3$ : $BCE = -\frac{1}{3}[\log(0.9) + \log(0.9) + \log(0.8)]$
The final output is the loss rounded to 4 decimal places: $BCE \approx 0.1446$

Constraints:

y_true: list of 0s and 1s
y_pred: list of predicted probabilities (0 to 1)
Clip predictions to [1e-7, 1-1e-7] before computing log
Return a single float rounded to 4 decimal places

More from Machine Learning 1

RBF Kernel MatrixHard Polynomial KernelMedium Hinge LossEasy Linear Decision BoundaryEasy Gradient Boosting Residual StepHard

Background Knowledge

The binary cross-entropy loss is a fundamental concept in machine learning, particularly in classification problems. It measures the difference between the predicted probabilities and the true labels. The goal is to minimize this loss function to achieve better predictions. The binary cross-entropy loss is defined as $BCE = -\frac{1}{n}\sum_{i=1}^{n}[y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)]$ , where $y_i$ represents the true labels and $\hat{y}_i$ represents the predicted probabilities.

In machine learning, loss functions are used to evaluate the performance of a model. The binary cross-entropy loss is commonly used in binary classification problems, where the target variable is binary (0 or 1). This loss function is also known as log loss. To avoid $\log(0)$ , which is undefined, predictions are clipped to the range $[\epsilon, 1-\epsilon]$ , where $\epsilon$ is a small value, typically $10^{-7}$ .

The binary cross-entropy loss is a differentiable function, which makes it suitable for optimization using gradient descent. The gradient descent algorithm iteratively updates the model's parameters to minimize the loss function. Understanding the binary cross-entropy loss and its properties is essential for building and optimizing machine learning models, especially in classification tasks.

Algorithm/Approach

The general approach to solving this problem involves implementing the binary cross-entropy loss formula. This requires iterating over the true labels and predicted probabilities, computing the loss for each sample, and summing up the losses. To avoid $\log(0)$ , the predictions need to be clipped to the range $[\epsilon, 1-\epsilon]$ . The final loss value is then rounded to 4 decimal places.

Step-by-Step Strategy

To implement the solution, follow these steps:

Clip the predicted probabilities to the range $[\epsilon, 1-\epsilon]$ to avoid $\log(0)$ .
Iterate over the true labels and clipped predicted probabilities.
For each sample, compute the loss using the binary cross-entropy loss formula: $y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i)$ .
Sum up the losses for all samples and divide by the total number of samples.
Round the final loss value to 4 decimal places.

Common Pitfalls

When implementing the solution, watch out for the following:

Make sure to clip the predicted probabilities to avoid $\log(0)$ .
Use the correct binary cross-entropy loss formula.
Iterate over the true labels and predicted probabilities correctly.
Round the final loss value to 4 decimal places as required.

Time & Space Complexity

The time complexity of the solution is O(n), where n is the number of samples, since we need to iterate over the true labels and predicted probabilities. The space complexity is O(1), since we only need to store the final loss value and a few intermediate variables. Note that the space complexity assumes that the input arrays (true labels and predicted probabilities) are already allocated and do not need to be stored in addition to the output.

solution.py

Python 3.13

Test Results

0/0

Run code to see test results.