Variational Autoencoders والفضاء الكامن

A variational autoencoder, or VAE, does not store one fixed code per input. Instead it learns a smooth, organized latent space you can actually sample from.

The VAE encoder outputs a mean and a variance instead of a single point. Each input becomes a little cloud of likely latent codes.

To get a code, you sample from that cloud using its mean and variance. Sampling adds the randomness that makes the space generative.

Random sampling blocks gradients, so the reparameterization trick moves the randomness aside. Now you can still backpropagate through the sample.

A VAE balances two goals: reconstruction loss for accurate rebuilds, plus a term that shapes the latent space. They pull against each other.

The second loss is the KL divergence. It nudges each cloud toward a standard normal, keeping the latent space tidy and continuous.

Thanks to the KL term, nearby codes decode to similar outputs. This smoothness means you can walk between points and get sensible results. ✨

Because the space is organized, you can sample new codes from a normal distribution and decode them into fresh, never-seen samples.

Blend two latent codes and decode the mix to morph one input into another. This smooth interpolation is a signature VAE trick.

A plain autoencoder compresses; a VAE compresses and becomes generative. The extra KL term is what unlocks sampling new data.

Weighting the KL term, often called beta, controls the tradeoff. Higher beta gives a cleaner space but slightly blurrier reconstructions.

A VAE encodes inputs as distributions, samples with the reparameterization trick, and adds a KL term for a smooth space you can sample and interpolate. 🎉