Compute-Bound vs Memory-Bound
Reading the roofline to plan fixes.
Compute-Bound vs Memory-Bound is a free CUDA Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the CUDA Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Two Kinds of Limit
Every kernel hits one of two walls. It is either compute-bound, limited by math, or memory-bound, limited by data movement. ⚖️
Compute-Bound, Defined
A compute-bound kernel keeps the math units busy and rarely waits on memory. Its limit is raw arithmetic throughput.
Memory-Bound, Defined
A memory-bound kernel spends its time waiting for data. The cores sit idle while bytes crawl in from global memory.
Arithmetic Intensity
The key ratio is arithmetic intensity: math operations done per byte loaded. High intensity leans compute, low leans memory.
The Roofline Picture
On a roofline plot, low-intensity kernels hit the sloped memory ceiling, while high-intensity ones hit the flat compute ceiling.
Why Diagnosis Matters
Fixing the wrong wall wastes effort. Adding math to a memory-bound kernel changes nothing; you must cut data traffic instead.
Fixing Memory-Bound Kernels
To speed a memory-bound kernel, coalesce accesses, reuse data in shared memory, and cache values to read less.
Fixing Compute-Bound Kernels
For compute-bound work, raise parallelism, use faster math, or reach for tensor cores to push past the math ceiling.
Most Kernels Are Memory-Bound
In practice, the majority of CUDA kernels are memory-bound. Bandwidth, not arithmetic, is usually the scarce resource.
Let the Profiler Decide
Do not guess the wall. The roofline in Nsight Compute places your kernel under the correct ceiling for you.
A Simple Mental Test
Ask one question: are the cores or the memory pipes closer to peak? Whichever is saturated names your bound.
Quick Check
A kernel has very low arithmetic intensity.
Recap
Diagnose the wall first: memory-bound kernels need less traffic, compute-bound ones need more math. The roofline tells you which. 👏
Frequently asked questions
Is the “Compute-Bound vs Memory-Bound” lesson free?
Yes — the full text of “Compute-Bound vs Memory-Bound” is free to read here on the web, and the CUDA Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the CUDA Academy course, upgrade to CoddyKit PRO.
What will I learn in “Compute-Bound vs Memory-Bound”?
Reading the roofline to plan fixes. You practise CUDA Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start CUDA Academy?
No prior experience is required. CUDA Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Compute-Bound vs Memory-Bound” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this CUDA Academy lesson?
Yes. Every CUDA Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Timeline View in Nsight Systems
- Kernel Metrics in Nsight Compute
- Compute-Bound vs Memory-Bound
- Annotating Code with NVTX