Freeze the Backbone, Train the Head
Reuse learned features cheaply.
Freeze the Backbone, Train the Head is a free Deep Learning Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Borrow a Smart Brain
Why train from zero? A model already trained on millions of images has learned useful features. Transfer learning reuses that knowledge for your task. 🧠
Backbone vs Head
A pretrained model has two parts: the backbone that extracts general features, and the head that makes the final prediction for its original task.
The Plan
The simplest recipe: freeze the backbone so it stays as is, then swap in a fresh head and train only that part on your own labels.
Load a Pretrained Model
Start with a model that already carries good weights. torchvision hands you popular architectures with one call.
from torchvision import models
model = models.resnet18(weights='DEFAULT')Freezing Means No Gradients
To freeze a layer you set requires_grad to False. PyTorch then skips computing gradients for it, so its weights never change.
for p in model.parameters():
p.requires_grad = FalseWhy Freeze at All?
Freezing the backbone keeps its hard-won features intact and means far fewer weights to update. That makes training faster and works with little data.
Swap the Head
The old head predicts the wrong classes. Replace it with a fresh Linear layer sized for your number of classes.
import torch.nn as nn
model.fc = nn.Linear(512, 3)The New Head Is Trainable
A brand-new layer starts with requires_grad set to True automatically. So only your head will learn while the frozen backbone watches.
Optimize Only the Head
Pass just the head's parameters to your optimizer. Updating only these few weights is what makes this approach so cheap.
opt = torch.optim.Adam(model.fc.parameters(), lr=1e-3)The Backbone Becomes an Extractor
With its weights frozen, the backbone simply turns each image into a rich feature vector. Your head learns to read those features.
When This Shines
Freeze-and-train-head is ideal with a small dataset or limited compute. You get strong results in minutes instead of days.
Quick Check
You froze the backbone and replaced the head. Which weights actually update during training?
Recap
You learned to borrow a pretrained backbone, freeze it, attach a fresh head, and train just that head. Fast, cheap, and great with small data. 🎯
Frequently asked questions
Is the “Freeze the Backbone, Train the Head” lesson free?
Yes — the full text of “Freeze the Backbone, Train the Head” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Freeze the Backbone, Train the Head”?
Reuse learned features cheaply. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Freeze the Backbone, Train the Head” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Freeze the Backbone, Train the Head
- Fine-Tune with a Lower Learning Rate
- Discriminative Layer-Wise Rates
- Fine-Tune a Hugging Face Model