Profile the Bottleneck
Find where time and memory go.
Profile the Bottleneck is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Profile First
Before optimizing, find out where time actually goes. Guessing wastes effort, while a quick profile shows you the real slow spots.
Two Common Bottlenecks
Training usually stalls in one of two places: the GPU compute doing math, or the data pipeline feeding it. Knowing which one matters.
Time It Crudely First
Start simple by timing a loop section with the clock. A rough perf_counter reading often points you to the right area in seconds.
import time
t = time.perf_counter()
# run one batch
print(time.perf_counter() - t)GPU Work Is Async
CUDA runs in the background, so naive timers lie. Call synchronize first to make sure the GPU has truly finished before you read the clock.
torch.cuda.synchronize()The Built-In Profiler
For real detail, use the torch.profiler context manager. It records how long every operation takes on both CPU and GPU.
from torch.profiler import profileWrap the Code to Profile
Run the part you care about inside a profile block. Choosing both CPU and CUDA activities captures the whole picture.
with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof:
model(x)Read the Table
Print results sorted by cost to see the heaviest ops at the top. The key_averages table groups identical operations together.
print(prof.key_averages().table(sort_by='cuda_time_total'))Spot a Data Bottleneck
If the GPU often sits idle waiting, your DataLoader is too slow. More workers or cached data usually fixes that gap.
Spot a Compute Bottleneck
If one matmul or conv dominates the table, the limit is raw compute. Mixed precision or a smaller model is the lever to pull.
Watch Memory Too
The profiler can also report peak memory. Tracking profile_memory reveals which layers eat the most, guiding what to trim.
with profile(profile_memory=True) as prof:
model(x)Measure, Change, Re-Measure
Optimization is a loop: profile, make one change, then profile again. Trust numbers, not hunches, to confirm a fix actually helped.
Quick Check
Your GPU often sits idle between batches. What is the likely bottleneck?
Recap
Profile before you tune: synchronize for honest timings, use torch.profiler to find the heaviest ops, then fix data or compute and measure again. 🔍
Frequently asked questions
Is the “Profile the Bottleneck” lesson free?
Yes — the full text of “Profile the Bottleneck” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Profile the Bottleneck”?
Find where time and memory go. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Profile the Bottleneck” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.