Sigmoid Gradient

easyFREEActivation FunctionsDeep LearningDeep Learning 1

Problem Statement

Compute the derivative of the Sigmoid function given the activation output.

Background

The Sigmoid function is defined as: $\sigma(z) = \frac{1}{1 + e^{-z}}$

A useful property is that its derivative can be computed directly from the output:

If $g = \sigma(z)$ , then the derivative $g'(z) = g \times (1 - g)$

This makes backpropagation efficient since we already have the sigmoid output from the forward pass.

Your Task

Write a function sigmoid_derivative(sigmoid_output) that returns the gradient values for each element.

Output Format

Return a list of floats representing the derivatives. Round each value to 4 decimal places.

Example

Input

sigmoid_output=[0.5, 0.8]

Output

[0.25, 0.16]

Reasoning

Input: sigmoid_output holds values $g = \sigma(z)$ that have already come out of the sigmoid in the forward pass, so each lies between 0 and 1. Here $g_1 = 0.5$ and $g_2 = 0.8$ ; the original $z$ values are never needed.
Math: each element is treated on its own with $\sigma'(z) = g(1 - g)$ .
For $g_1 = 0.5$ : $1 - g_1 = 0.5$ , so $0.5 \times 0.5 = 0.25$ .
For $g_2 = 0.8$ : $1 - g_2 = 0.2$ , so $0.8 \times 0.2 = 0.16$ , which stays $0.16$ after rounding to 4 decimal places.
Output: [0.25, 0.16], a list with one gradient per input value and the same length as the input.
Notice: $0.25$ is the largest value the derivative can ever take, reached at $g = 0.5$ (the sigmoid's steepest point, $z = 0$ ). As $g$ approaches 0 or 1 the gradient shrinks toward 0, which is why saturated sigmoids learn slowly. The formula is also symmetric, so $g$ and $1 - g$ give the same gradient: $0.25$ and $0.75$ both give $0.1875$ .

Constraints

0 <= sigmoid_output[i] <= 1 for each element
List length: 1 to 100 elements

Learn this in

ClassificationML Study Plan Neural NetworksML Study Plan

More from Deep Learning 1

Early Stopping Checkeasy L2 Regularization Penaltyeasy Inverted Dropout Forwardmedium Binary Cross-Entropy Losseasy Momentum Update Stepmedium

Concept primer

1. Background Knowledge

The sigmoid function is a fundamental activation function in neural networks, defined as: $\sigma(z) = \frac{1}{1 + e^{-z}}$

It maps any real input $z \in \mathbb{R}$ to the range $(0, 1)$ , making it ideal for binary classification and probability outputs. Key properties include:

Smooth and differentiable everywhere
S-shaped (S-curve) with inflection point at $z = 0$ , where $\sigma(0) = 0.5$
Asymptotic behavior: $\sigma(z) \to 1$ as $z \to \infty$ , $\sigma(z) \to 0$ as $z \to -\infty$

Critical property for backpropagation: The derivative can be computed directly from the output: $\sigma'(z) = \sigma(z) \cdot (1 - \sigma(z))$

Proof sketch: Let g = \sigma(z). Then: $\sigma'(z) = \frac{d}{dz} \left( \frac{1}{1 + e^{-z}} \right) = \frac{e^{-z}}{(1 + e^{-z})^2} = \frac{1}{1 + e^{-z}} \cdot \frac{e^{-z}}{1 + e^{-z}} = g \cdot (1 - g)$

This eliminates the need to store $z$ values during the forward pass, saving memory in deep networks.

Prerequisites:

Basic calculus (chain rule, quotient rule)
Understanding of neural network backpropagation
Python list operations and rounding

2. Algorithm Approach

Direct computation using the closed-form derivative formula—no numerical differentiation needed:

For each element $g_i$ in the input list, compute $g_i \times (1 - g_i)$
Round result to 4 decimal places
Return as list

Alternative approaches (not needed here but educational):

Numerical approximation: $\sigma'(z) \approx \frac{$ \sigma $(z + h) -$ \sigma $(z - h)}{2h}$ (less efficient, error-prone)
Vectorized operations (NumPy preferred for larger arrays)

Time complexity: $O(n)$ where $n$ is list length (1-100 elements).

3. Step-by-Step Strategy

Input validation (optional but good practice): Ensure $0 \leq g_i \leq 1$
Element-wise computation: For each $g$ in sigmoid_output, calculate $g \times (1 - g)$
Precision handling: Use round(value, 4) for each result
List construction: Collect results in new list
Return: Output list matching input length

Sample implementation:

def sigmoid_derivative(sigmoid_output):
 return [round(g * (1 - g), 4) for g in sigmoid_output]

Verification:

Input [0.5, 0.8] → [0.5×0.5=0.25, 0.8×0.2=0.16] ✓
Edge cases: [0.0] → [0.0], [1.0] → [0.0], maximum at [0.5] → [0.25]

4. Common Pitfalls

Not rounding to 4 decimals: 0.159999999 instead of 0.1600

# Wrong
[g * (1 - g) for g in sigmoid_output] # [0.16] may print as [0.159999999]

# Correct
[round(g * (1 - g), 4) for g in sigmoid_output] # [0.16]

Input range violation: Derivative undefined or negative outside $[0,1]$

sigmoid_derivative([-0.1, 1.1]) # Produces negative values—invalid!

Using sigmoid input instead of output: Must use activation output $g$ , not pre-activation $z$
Floating-point precision: Use round() consistently; Python's print may show 0.16 as 0.159999999
Mutable input modification: Don't alter original list—create new one

5. Time & Space Complexity

Aspect	Complexity	Explanation
Time	O(n)	Single pass over n elements (1 ≤ n ≤ 100); constant-time arithmetic per element
Space	O(n)	Output list stores n floats; no additional data structures needed

Scalability notes:

Perfectly efficient for constraints (n ≤ 100)
For large arrays, use NumPy: np.round(sigmoid_output * (1 - sigmoid_output), 4)
No recursion, sorting, or heavy computation—purely linear

Gradient interpretation: Maximum gradient magnitude is $0.25$ at $g = 0.5$ , vanishing at boundaries ( $g=0,1$ )—explains vanishing gradient problem in deep sigmoid networks.

solution.py

Python 3.13

Test results