U-Net으로 잡음 예측하기

At every reverse step a single network does the heavy lifting. Its job is simple to state: look at a noisy image and predict the noise inside it.

Diffusion almost always uses a U-Net, because its output is the same size as its input: a noisy image in, a noise map out.

The encoder downsamples the image, capturing big-picture structure like shapes and layout at a small resolution.

The decoder then upsamples back to full size, rebuilding fine detail so the predicted noise lines up pixel for pixel.

The U shape comes from skip connections that pass detail from encoder to decoder, so sharp edges survive the squeeze.

The U-Net also needs the timestep t, so it knows whether to remove a lot of noise or just a little at this stage.

That timestep is turned into a vector called a time embedding and mixed into the network's layers as a conditioning signal.

Many U-Nets add attention at the lowest resolution, letting distant regions of the image coordinate their content.

Training compares the predicted noise to the real noise with simple MSE loss. That single objective is all it takes.

One U-Net handles all timesteps. The t input is what tells it how aggressively to denoise right now.

Predict noise, subtract a slice of it, feed the result back in. Repeating this is how the U-Net walks toward a clean image. 🖼️

What role do the U-Net's skip connections play?

A U-Net predicts noise at every step, using skip connections for detail and a time embedding to gauge the timestep. One MSE loss trains it all. 🎉