Implement Leaky ReLU
Problem Statement
Implement the Leaky ReLU activation function.
Background
Leaky ReLU fixes the "dying ReLU" problem by allowing a small gradient for negative inputs:
- f(x)=x if x>0
- f(x)=α⋅x if x≤0
where α is a small constant (commonly 0.01).
Your Task
Write a function leaky_relu(values, alpha=0.01) that applies Leaky ReLU element-wise to a list of values.
Output Format
Return a list of floats with Leaky ReLU applied to each element. Round each value to 4 decimal places.
Example:
values=[-1.0, 2.0], alpha=0.1
[-0.1, 2.0]
For -1.0: since -1.0 <= 0, apply alpha: 0.1 × (-1.0) = -0.1. For 2.0: since 2.0 > 0, keep as is: 2.0.
Constraints:
- -1000 <= x <= 1000 for each element
- 0 < alpha <= 1
- List length: 1 to 100 elements
1. Background Knowledge
Leaky ReLU addresses the dying ReLU problem in neural networks, where standard ReLU (f(x)=max(0,x)) outputs zero for negative inputs, causing gradients to vanish and neurons to "die" during backpropagation. Leaky ReLU modifies this by allowing a small non-zero gradient for negative inputs:
f(x)={xα⋅xif x>0if x≤0where α (typically 0.01) is a leakage parameter ensuring gradient flow. This maintains ReLU's computational efficiency while preventing neuron death, improving training stability in deep networks.
Prerequisites: Basic Python list operations, conditional logic, and understanding of element-wise operations in ML.
2. Algorithm Approach
Use element-wise conditional application—the standard technique for activation functions:
- Iterate through input list
- Apply piecewise function based on sign of each element
- Round results to 4 decimal places
NumPy alternative (for reference, not required): np.maximum(values, alpha * np.array(values)), but pure Python needed here.
Mathematical foundation: Piecewise linear function, preserving differentiability except at x=0 (subgradient exists).
3. Step-by-Step Strategy
def leaky_relu(values, alpha=0.01):
result = []
for x in values:
if x > 0:
result.append(round(x, 4))
else:
result.append(round(alpha * x, 4))
return result
Continue the full explanation
You're reading the free preview. Unlock the complete walkthrough, the code editor, test runner and reference solution with Premium.
Editor locked
The code editor is locked for Pro problems. It is only available for free problems. Please upgrade to gain access to the code editor for all problems.