Implement Leaky ReLU

easyPROActivation FunctionsDeep LearningDeep Learning 1

Problem Statement

Implement the Leaky ReLU activation function.

Background

Leaky ReLU fixes the "dying ReLU" problem by allowing a small gradient for negative inputs:

$f(x) = x$ if $x > 0$
$f(x) = \alpha \cdot x$ if $x \leq 0$

where $\alpha$ is a small constant (commonly 0.01).

Your Task

Write a function leaky_relu(values, alpha=0.01) that applies Leaky ReLU element-wise to a list of values.

Output Format

Return a list of floats with Leaky ReLU applied to each element. Round each value to 4 decimal places.

Example

Input

values=[-1.0, 2.0], alpha=0.1

Output

[-0.1, 2.0]

Reasoning

Inputs: values is a list of real numbers, here $[-1.0,\ 2.0]$ , and alpha $=0.1$ is the slope used on the negative side (it overrides the default of $0.01$ ).
Element $-1.0$ : it is not positive, so the leaky branch applies: $f(-1.0) = \alpha \cdot (-1.0) = 0.1 \times (-1.0) = -0.1$ . The value is shrunk tenfold but keeps its sign instead of becoming $0$ .
Element $2.0$ : it is positive, so the function is the identity: $f(2.0) = 2.0$ .
Output: [-0.1, 2.0], one float per input, in the same order, each rounded to 4 decimal places (nothing here needs trimming).
Notice: $x = 0$ falls in the $x \le 0$ case, giving $\alpha \cdot 0 = 0.0$ . With the default $\alpha = 0.01$ , an input of $-5.0$ would become $-0.05$ . Plain ReLU would have sent it to $0$ .

Constraints

-1000 <= x <= 1000 for each element
0 < alpha <= 1
List length: 1 to 100 elements

Learn this in

Neural NetworksML Study Plan

More from Deep Learning 1

Early Stopping Checkeasy L2 Regularization Penaltyeasy Inverted Dropout Forwardmedium Binary Cross-Entropy Losseasy Momentum Update Stepmedium

Concept primerPREVIEW

1. Background Knowledge

Leaky ReLU addresses the dying ReLU problem in neural networks, where standard ReLU ( $f(x) = \max(0, x)$ ) outputs zero for negative inputs, causing gradients to vanish and neurons to "die" during backpropagation. Leaky ReLU modifies this by allowing a small non-zero gradient for negative inputs:

f(x) = \begin{cases} x & \text{if } x > 0 \\ \alpha \cdot x & \text{if } x \leq 0 \end{cases}

where $\alpha$ (typically 0.01) is a leakage parameter ensuring gradient flow. This maintains ReLU's computational efficiency while preventing neuron death, improving training stability in deep networks.

Prerequisites: Basic Python list operations, conditional logic, and understanding of element-wise operations in ML.

2. Algorithm Approach

Use element-wise conditional application—the standard technique for activation functions:

Iterate through input list
Apply piecewise function based on sign of each element
Round results to 4 decimal places

NumPy alternative (for reference, not required): np.maximum(values, alpha * np.array(values)), but pure Python needed here.

Mathematical foundation: Piecewise linear function, preserving differentiability except at $x=0$ (subgradient exists).

3. Step-by-Step Strategy

def leaky_relu(values, alpha=0.01):
 result = []
 for x in values:
 if x > 0:
 result.append(round(x, 4))
 else:
 result.append(round(alpha * x, 4))
 return result

PRO

Continue the full explanation

You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.

PRO

Editor locked

The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.

solution.py

Python 3.13

Test results