0Pricing
Learn AI with Python · Lesson

Autoencoders for Representation Learning

Encoder-decoder architecture, bottleneck, reconstruction loss, applications in anomaly detection.

Autoencoders for Representation Learning is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What is an Autoencoder?

An autoencoder learns to compress data into a small code and reconstruct it back. It has two halves: an encoder that compresses, and a decoder that rebuilds. The squeeze in the middle forces it to learn what matters.

import torch
import torch.nn as nn

The Bottleneck Idea

The narrow middle layer is the bottleneck or latent code. By forcing all information through a few numbers, the network must discard noise and keep the essential structure of the data.

Building the Encoder

For 28x28 images (784 pixels) the encoder shrinks the input step by step: 784 to 256 to a 64-dim latent code. Each Linear layer is followed by a ReLU nonlinearity.

encoder = nn.Sequential(
    nn.Linear(784, 256),
    nn.ReLU(),
    nn.Linear(256, 64)
)

Building the Decoder

The decoder mirrors the encoder, expanding the 64-dim code back to 784 pixels. A final Sigmoid squashes outputs to [0,1] to match normalized pixel values.

decoder = nn.Sequential(
    nn.Linear(64, 256),
    nn.ReLU(),
    nn.Linear(256, 784),
    nn.Sigmoid()
)

The Full Autoencoder

Wire encoder and decoder into one module. The forward pass encodes then decodes, returning a reconstruction the same shape as the input.

class AutoEncoder(nn.Module):
    def __init__(self, enc, dec):
        super().__init__()
        self.enc, self.dec = enc, dec
    def forward(self, x):
        z = self.enc(x)
        return self.dec(z)

Reconstruction Loss (MSE)

The training target is the input itself. Mean Squared Error measures how far the reconstruction is from the original pixel by pixel. Minimizing it teaches faithful compression.

criterion = nn.MSELoss()

x_flat = x.view(x.size(0), -1)
recon = model(x_flat)
loss = criterion(recon, x_flat)

Training the Autoencoder

Training is unsupervised: no labels, only inputs. The loop is the standard zero_grad, forward, backward, step, but the loss compares the output to the input.

optimizer.zero_grad()
recon = model(x_flat)
loss = criterion(recon, x_flat)
loss.backward()
optimizer.step()

The Latent Space

After training, the encoder maps each input to a point in the latent space. Similar inputs land near each other, so this 64-dim code is a learned, compact representation of the data.

z = model.enc(x_flat)
print(z.shape)  # [batch, 64]

Visualizing with t-SNE

The 64-dim latent space is hard to see. t-SNE projects it to 2D for plotting. Clusters in the plot reveal that the autoencoder grouped similar items together without ever seeing labels.

from sklearn.manifold import TSNE

codes = model.enc(all_data).detach().numpy()
proj = TSNE(n_components=2).fit_transform(codes)
# scatter plot proj colored by true label

Anomaly Detection

An autoencoder trained on normal data reconstructs it well but struggles with anomalies it never saw. A high reconstruction error flags an outlier, a powerful unsupervised anomaly detector.

errors = ((model(data) - data) ** 2).mean(dim=1)
anomalies = errors > threshold

Why Representation Learning?

The real prize is the latent code: a learned feature vector you can use for clustering, visualization, anomaly detection, or as input to another model, all learned without labels.

Quick Check

Test your autoencoder understanding.

Recap: Autoencoders

You built an encoder (784 to 256 to 64) and a mirrored decoder, trained with MSE reconstruction loss and no labels. You used the latent code for representation learning, visualized it with t-SNE, and detected anomalies via high reconstruction error.

Frequently asked questions

Is the “Autoencoders for Representation Learning” lesson free?

Yes — the full text of “Autoencoders for Representation Learning” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.

What will I learn in “Autoencoders for Representation Learning”?

Encoder-decoder architecture, bottleneck, reconstruction loss, applications in anomaly detection. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Learn AI with Python?

No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Autoencoders for Representation Learning” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Learn AI with Python lesson?

Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Autoencoders for Representation Learning
  2. Variational Autoencoders (VAE)
  3. GANs: Generator and Discriminator
  4. Conditional GANs and Style Transfer
← Back to Learn AI with Python