ResNet: 스킵 연결로 깊어지기

Surprisingly, plain nets past a point trained worse as they got deeper. More layers should not hurt, yet they did. Something was broken.

Gradients shrank as they flowed back through many layers, a problem called vanishing gradients. Deep nets stopped learning.

ResNets fix is elegant: add a shortcut that lets the input jump past a block and add itself to the output.

Instead of learning the full mapping, each block only learns the residual, the small change to add on top of the input.

If a block is not useful, it can just learn zeros and pass the input straight through. Doing nothing becomes effortless.

The shortcut gives gradients a clean path backward, so even very deep networks keep learning.

A block computes some layers, then adds the original input back. That single add is the whole trick.

With shortcuts, ResNet trained 50, 101, even 152 layers and won ImageNet 2015. Depth finally paid off.

Deeper ResNets use a 1x1, 3x3, 1x1 bottleneck to cut compute while keeping representational power.

This residual idea spread far beyond vision. Transformers and many modern nets rely on the same identity shortcut.

You can sketch the core in a few lines: convolve, then add the input back before the activation. Here is the shape of it.

ResNets skip connections made extreme depth trainable by letting blocks learn residuals. One add changed everything. Great progress!