Mode Collapse & Stabilizing Tricks

GANs are powerful but notoriously fragile. Two networks chasing each other can swing, stall, or collapse. Knowing the failure modes saves you hours. 🧯

Mode collapse happens when the generator produces only a few outputs that reliably fool the judge, ignoring the full variety of the real data.

You spot it when generated samples all look nearly identical, even though your real dataset is rich and diverse. Variety has vanished.

If the discriminator gets too strong too quickly, its gradients dry up and the generator stops learning. Balance between the two is everything.

Label smoothing softens the real target from 1.0 to about 0.9. This keeps the discriminator humble and its gradients useful.

Adding a little noise to the discriminator's inputs makes its job slightly harder, which steadies the training game and reduces collapse.

The DCGAN paper uses Adam with a low learning rate and beta1 set to 0.5. These settings are a reliable starting point.

Minibatch features let the discriminator compare samples within a batch. If they all look the same, it notices, punishing collapse directly.

The Wasserstein loss with gradient penalty gives smoother signals than plain cross-entropy, making notoriously unstable GANs far easier to train.

Aim for a fair fight. If one network dominates, slow it down or update the other more often. Balance beats any single trick.

A healthy GAN makes images that are both sharp and varied. Always check that different latent vectors give genuinely different outputs.

Recognizing the symptom is half the cure.

You learned to spot mode collapse and to fight it with label smoothing, input noise, tuned Adam, and a balanced generator-discriminator duo. 💪