Variational Autoencoders & the Latent Space
Sample new data from a learned distribution.
Variational Autoencoders & the Latent Space is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Beyond Plain Codes
A variational autoencoder, or VAE, does not store one fixed code per input. Instead it learns a smooth, organized latent space you can actually sample from.
Encode to a Distribution
The VAE encoder outputs a mean and a variance instead of a single point. Each input becomes a little cloud of likely latent codes.
mu, logvar = encoder(x)Sample From the Cloud
To get a code, you sample from that cloud using its mean and variance. Sampling adds the randomness that makes the space generative.
The Reparameterization Trick
Random sampling blocks gradients, so the reparameterization trick moves the randomness aside. Now you can still backpropagate through the sample.
std = torch.exp(0.5 * logvar)
z = mu + std * torch.randn_like(std)Two Losses, Not One
A VAE balances two goals: reconstruction loss for accurate rebuilds, plus a term that shapes the latent space. They pull against each other.
The KL Divergence Term
The second loss is the KL divergence. It nudges each cloud toward a standard normal, keeping the latent space tidy and continuous.
kl = -0.5 * torch.sum(1 + logvar - mu**2 - logvar.exp())A Smooth Latent Space
Thanks to the KL term, nearby codes decode to similar outputs. This smoothness means you can walk between points and get sensible results. ✨
Generate Brand-New Data
Because the space is organized, you can sample new codes from a normal distribution and decode them into fresh, never-seen samples.
z = torch.randn(1, latent_dim)
new_sample = decoder(z)Interpolate Between Inputs
Blend two latent codes and decode the mix to morph one input into another. This smooth interpolation is a signature VAE trick.
z = 0.5 * z_a + 0.5 * z_bVAE vs Plain Autoencoder
A plain autoencoder compresses; a VAE compresses and becomes generative. The extra KL term is what unlocks sampling new data.
Tune the Balance
Weighting the KL term, often called beta, controls the tradeoff. Higher beta gives a cleaner space but slightly blurrier reconstructions.
Quick Check
What gives a VAE its generative power?
Recap
A VAE encodes inputs as distributions, samples with the reparameterization trick, and adds a KL term for a smooth space you can sample and interpolate. 🎉
Frequently asked questions
Is the “Variational Autoencoders & the Latent Space” lesson free?
Yes — the full text of “Variational Autoencoders & the Latent Space” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Variational Autoencoders & the Latent Space”?
Sample new data from a learned distribution. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Variational Autoencoders & the Latent Space” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Encoder, Bottleneck & Decoder
- Denoising Autoencoders
- Variational Autoencoders & the Latent Space
- Anomaly Detection by Reconstruction Error