더 낮은 학습률로 미세 조정하기

Freezing the backbone is fast, but sometimes you want the whole model to adapt. Fine-tuning unfreezes it and trains everything together. 🔧

To fine-tune, you flip requires_grad back to True so the pretrained weights can move again during training.

Those pretrained weights are already excellent. A large learning rate would overwrite them with noisy updates and destroy what they learned.

The fix is a lower learning rate. Small steps nudge the pretrained weights toward your task without erasing their knowledge.

A common rule of thumb: use a learning rate ten to a hundred times smaller than you would for training from scratch.

A popular trick: train only the head for a few epochs, then unfreeze and fine-tune. This avoids a shock to the backbone early on.

Fine-tuning can quickly overfit a small dataset. Keep an eye on validation loss and stop as soon as it starts climbing.

Because the model already understands the data, fine-tuning usually converges in just a few epochs, not dozens.

A scheduler can shrink the learning rate further over time, letting the model settle gently into a good solution.

Full fine-tuning needs more data and compute but reaches higher accuracy. Frozen training is cheaper. The tradeoff depends on your dataset size.

Since fine-tuning can drift, save the model whenever validation accuracy improves so you never lose your best version.

You unfroze the backbone, fine-tuned with a low learning rate, warmed up the head first, and watched validation to stop overfitting. 🎯