VAE Reparameterization Trick

mediumPROGenerative ModelsMachine Learning 2

Implement the reparameterization trick used in Variational Autoencoders.

The encoder outputs mean $\mu$ and log-variance $\log\sigma^2$ for each latent dimension. To sample $z$ in a differentiable way:

where $\epsilon$ is a random sample from $\mathcal{N}(0, 1)$ .

Given $\mu$ , $\log\sigma^2$ , and pre-generated $\epsilon$ values, compute the latent sample $z$ .

Return $z$ as a list rounded to 4 decimal places.

Input

mu = [0.5, -0.3]
log_var = [0.0, 0.0]
epsilon = [1.0, -1.0]

Output

[1.5, -1.3]

Reasoning

Inputs: each list has one entry per latent dimension, here 2. mu holds the encoder's means $\mu = (0.5, -0.3)$ , log_var holds $\log\sigma^2 = (0, 0)$ , and epsilon holds noise already drawn from $\mathcal{N}(0, 1)$ : $\epsilon = (1.0, -1.0)$ .
Spread: a log-variance of $0$ means $\sigma^2 = 1$ , so $\sigma = e^{0/2} = e^0 = 1$ in both dimensions. The noise is passed through unscaled.
Shift and scale: dimension 1 gives $z_1 = 0.5 + 1 \cdot 1.0 = 1.5$ ; dimension 2 gives $z_2 = -0.3 + 1 \cdot (-1.0) = -1.3$ .
Output: [1.5, -1.3], one $z$ per dimension, same length as the inputs. Both values already have fewer than 4 decimal places, so rounding leaves them as is.
Notice: $z$ is a deterministic function of $\mu$ and $\sigma$ once $\epsilon$ is fixed, which is why gradients can flow through it. With $\log\sigma^2 = 2$ the spread becomes $\sigma = e^{1} \approx 2.7183$ , so $\mu = 1.0$ , $\epsilon = 0.5$ gives $z \approx 1.0 + 1.3591 = 2.3591$ .

Background Knowledge