Transfer Learning Concepts and Strategies
Feature extraction vs fine-tuning, frozen layers, when transfer learning helps.
Transfer Learning Concepts and Strategies is a free Learn AI with Python lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Learn AI with Python learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What Is Transfer Learning?
Transfer learning reuses a model trained on a large dataset as the starting point for a new, related task. Instead of learning visual features from scratch, you borrow features a network already learned.
This dramatically cuts the data and compute you need to reach high accuracy.
The ImageNet Foundation
Most pretrained vision models are trained on ImageNet: roughly 1.4 million images across 1000 classes.
During this training the network learns a hierarchy of features: edges and colors in early layers, textures in the middle, and object parts near the top. These low-level features transfer well to almost any image task.
Why Early Features Transfer
The first convolutional layers detect generic patterns (edges, blobs, gradients) that appear in every natural image, whether cats, X-rays, or satellite photos.
The deeper layers become more task-specific (they assemble those edges into ImageNet object parts). That is why we usually keep early layers and replace or retrain the later ones.
Strategy 1: Feature Extraction
Feature extraction freezes every layer of the pretrained base so its weights never change. The base acts as a fixed feature extractor; you only train a small new classification head on top.
Use this when your dataset is small and similar to ImageNet. It is fast and resists overfitting because few parameters are trainable.
base.trainable = False
model = Sequential([
base, # frozen ImageNet features
GlobalAveragePooling2D(),
Dense(num_classes, activation="softmax") # only this trains
])Strategy 2: Fine-Tuning
Fine-tuning unfreezes the top layers of the base and retrains them together with the head, letting the network adapt its high-level features to your specific data.
Use this when you have more data or your domain differs from ImageNet (e.g. medical scans). Always fine-tune at a very low learning rate to avoid destroying the pretrained weights.
Feature Extraction vs Fine-Tuning
- Feature extraction: base frozen, only head trains, very fast, best for small/similar data.
- Fine-tuning: top base layers unfrozen + head, slower, best for larger/dissimilar data.
A common pattern is to do feature extraction first, then fine-tune in a second phase once the head has stabilized.
Data Volume Requirements
How much data you have decides your strategy:
- Few hundred images: feature extraction only (freeze everything).
- A few thousand: feature extraction, then fine-tune the top few layers.
- Tens of thousands+: fine-tune larger portions of the base, possibly the whole network.
Less data means freeze more, because fewer trainable parameters reduce overfitting risk.
Replacing the Classification Head
ImageNet models output 1000 classes. Your task probably has a different number, so you remove the original top (include_top=False) and attach your own head sized to your classes.
from tensorflow.keras import layers, Model
x = base.output
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.2)(x)
outputs = layers.Dense(5, activation="softmax")(x) # 5 custom classes
model = Model(base.input, outputs)Preprocessing Must Match
Each pretrained family expects inputs scaled the same way they were during ImageNet training. Using the matching preprocess_input is essential or accuracy collapses.
from tensorflow.keras.applications.resnet50 import preprocess_input
# Scale pixels exactly as ResNet50 expects
x = preprocess_input(raw_image_batch)Learning Rate Discipline
During feature extraction a normal learning rate (e.g. 1e-3) is fine because only the fresh head trains.
During fine-tuning you must drop to something tiny like 1e-5. Large updates would wipe out the carefully learned ImageNet weights, an effect called catastrophic forgetting.
A Typical Workflow
- Load a pretrained base with
include_top=False. - Freeze it and train a new head (feature extraction).
- Unfreeze the top layers and retrain at a tiny learning rate (fine-tuning).
- Evaluate and convert for deployment.
This staged approach gives strong accuracy even with modest datasets.
Quick Check
Test your understanding of transfer learning strategies.
Recap
You learned that transfer learning reuses ImageNet-pretrained features (1.4M images, 1000 classes). Feature extraction freezes the base for small/similar datasets; fine-tuning unfreezes top layers at a tiny learning rate for larger/dissimilar data.
Match preprocessing to the base, replace the head for your classes, and use a staged workflow. Next we put this into code with VGG16 and ResNet50.
Frequently asked questions
Is the “Transfer Learning Concepts and Strategies” lesson free?
Yes — the full text of “Transfer Learning Concepts and Strategies” is free to read here on the web, and the Learn AI with Python course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Learn AI with Python course, upgrade to CoddyKit PRO.
What will I learn in “Transfer Learning Concepts and Strategies”?
Feature extraction vs fine-tuning, frozen layers, when transfer learning helps. You practise Learn AI with Python with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Learn AI with Python?
No prior experience is required. Learn AI with Python on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Transfer Learning Concepts and Strategies” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Learn AI with Python lesson?
Yes. Every Learn AI with Python lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Transfer Learning Concepts and Strategies
- Using VGG16 and ResNet50 as Base Models
- Fine-tuning: Unfreezing and Retraining
- MobileNet and EfficientNet for Edge Deployment