Deep Learning Academy · 课时

使用更低的学习率微调

缓慢更新预训练权重

使用更低的学习率微调 是 CoddyKit 上的免费 Deep Learning Academy 课时。这是第 2 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Deep Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Deep Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Beyond Just the Head

Freezing the backbone is fast, but sometimes you want the whole model to adapt. Fine-tuning unfreezes it and trains everything together. 🔧

Unfreeze the Backbone

To fine-tune, you flip requires_grad back to True so the pretrained weights can move again during training.

for p in model.parameters():
    p.requires_grad = True

The Danger of a Big Step

Those pretrained weights are already excellent. A large learning rate would overwrite them with noisy updates and destroy what they learned.

Go Gentle

The fix is a lower learning rate. Small steps nudge the pretrained weights toward your task without erasing their knowledge.

opt = torch.optim.Adam(model.parameters(), lr=1e-5)

How Much Lower?

A common rule of thumb: use a learning rate ten to a hundred times smaller than you would for training from scratch.

Warm Up the Head First

A popular trick: train only the head for a few epochs, then unfreeze and fine-tune. This avoids a shock to the backbone early on.

Watch Validation Loss

Fine-tuning can quickly overfit a small dataset. Keep an eye on validation loss and stop as soon as it starts climbing.

Fewer Epochs Needed

Because the model already understands the data, fine-tuning usually converges in just a few epochs, not dozens.

Pair It With a Scheduler

A scheduler can shrink the learning rate further over time, letting the model settle gently into a good solution.

sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=5)

Full Fine-Tune vs Frozen

Full fine-tuning needs more data and compute but reaches higher accuracy. Frozen training is cheaper. The tradeoff depends on your dataset size.

Keep the Best Checkpoint

Since fine-tuning can drift, save the model whenever validation accuracy improves so you never lose your best version.

Quick Check

You unfroze a pretrained model to fine-tune it. What learning rate should you use?

Recap

You unfroze the backbone, fine-tuned with a low learning rate, warmed up the head first, and watched validation to stop overfitting. 🎯

常见问题解答

「使用更低的学习率微调」课时是免费的吗？

是的 — 「使用更低的学习率微调」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Deep Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Deep Learning Academy 课程共包含 4 节课。

「使用更低的学习率微调」这节课中我会学到什么？

缓慢更新预训练权重你通过在浏览器中直接运行的动手代码来练习 Deep Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Deep Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Deep Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 2 节课，共 4 节。

「使用更低的学习率微调」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Deep Learning Academy 课中编写并运行代码吗？

能。每节 Deep Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Deep Learning Academy