Hinge Loss

easyFREESupport Vector MachinesMachine Learning 1

Compute the hinge loss for SVM classification.

Given true labels $y_i \in \{-1, +1\}$ and raw predictions (scores) $\hat{y}_i$ :

$L = \frac{1}{n}\sum_{i=1}^{n} \max(0, 1 - y_i \cdot \hat{y}_i)$

This is the loss function used by Support Vector Machines. Points correctly classified with margin $\geq 1$ contribute zero loss.

Return the average hinge loss rounded to 4 decimal places.

Example

Input

y_true = [1, -1, 1, -1]
y_scores = [0.8, -1.2, 0.3, -0.5]

Output

0.35

Reasoning

Inputs: y_true holds the four true labels $(+1, -1, +1, -1)$ and y_scores holds the classifier's raw scores $(0.8, -1.2, 0.3, -0.5)$ . The sign of a score is the predicted class; its size is the confidence.
Margins: multiply each label by its score, $m_i = y_i \hat{y}_i$ , giving $0.8,\ 1.2,\ 0.3,\ 0.5$ . All four are positive, so every point is on the correct side.
Per-point loss: $\max(0, 1 - m_i)$ gives $0.2,\ 0,\ 0.7,\ 0.5$ . The second point has margin $1.2 \geq 1$ , so $1 - 1.2 = -0.2$ is clipped to $0$ .
Average: $L = \frac{0.2 + 0 + 0.7 + 0.5}{4} = \frac{1.4}{4} = 0.35$ , which is the output 0.35.
Notice: three points are classified correctly yet still pay a penalty, because their margin is below $1$ . Hinge loss rewards confident correct predictions, not just correct ones. A wrong sign (negative margin) would push a point's loss above $1$ .

Constraints

y_true: list of labels in {-1, +1}
y_scores: list of raw prediction scores (any real number)
Return average hinge loss rounded to 4 decimal places

Learn this in

Support Vector MachinesML Study Plan

More from Machine Learning 1

RBF Kernel Matrixhard Polynomial Kernelmedium Linear Decision Boundaryeasy Gradient Boosting Residual Stephard Random Feature Subspacemedium

Concept primer

Background Knowledge

The hinge loss is a common loss function used in Support Vector Machines (SVMs) for classification problems. In SVMs, the goal is to find a hyperplane that maximally separates the classes in the feature space. The hinge loss measures the difference between the predicted score and the true label. It is defined as $L = \max(0, 1 - y_i \cdot \hat{y}_i)$ , where $y_i$ is the true label and $\hat{y}_i$ is the predicted score. The hinge loss is zero if the point is correctly classified with a margin of at least 1.

The margin is a key concept in SVMs, which refers to the distance between the hyperplane and the nearest data point. A larger margin indicates a more confident classification. The hinge loss encourages the model to maximize the margin by penalizing points that are not correctly classified with a sufficient margin. The raw predictions or scores are the output of the SVM model before applying the sign function to obtain the final predicted label. These scores represent the distance from the hyperplane, and the sign of the score indicates the predicted class.

In the context of this problem, we are given the true labels $y_i$ and the raw predictions $\hat{y}_i$ , and we need to compute the average hinge loss over all data points. This requires understanding the hinge loss formula and how to implement it in code. The formula involves a summation over all data points, and we need to round the final result to 4 decimal places.

Algorithm/Approach

The general approach to solving this problem involves iterating over all data points, computing the hinge loss for each point, and then averaging the losses. This can be achieved using a simple loop or vectorized operations, depending on the programming language and the size of the dataset. The key is to correctly implement the hinge loss formula and handle the cases where the point is correctly classified with a margin of at least 1.

Step-by-Step Strategy

To implement the solution, follow these steps:

Initialize a variable to store the sum of hinge losses
Iterate over all data points, computing the hinge loss for each point using the formula $L = \max(0, 1 - y_i \cdot \hat{y}_i)$
Add the hinge loss for each point to the running sum
Divide the sum by the total number of data points to obtain the average hinge loss
Round the average hinge loss to 4 decimal places

Common Pitfalls

When implementing the solution, watch out for the following common pitfalls:

Incorrectly handling the cases where the point is correctly classified with a margin of at least 1
Failing to round the final result to 4 decimal places
Using the wrong formula for the hinge loss
Not initializing the sum of hinge losses to zero

Time & Space Complexity

The time complexity of the solution is O(n), where n is the number of data points, since we need to iterate over all points to compute the hinge loss. The space complexity is O(1), since we only need to store the sum of hinge losses and the average hinge loss, regardless of the size of the dataset.

solution.py

Python 3.13

Test results