مقيّد بالحساب مقابل مقيّد بالذاكرة

Every kernel hits one of two walls. It is either compute-bound, limited by math, or memory-bound, limited by data movement. ⚖️

A compute-bound kernel keeps the math units busy and rarely waits on memory. Its limit is raw arithmetic throughput.

A memory-bound kernel spends its time waiting for data. The cores sit idle while bytes crawl in from global memory.

The key ratio is arithmetic intensity: math operations done per byte loaded. High intensity leans compute, low leans memory.

On a roofline plot, low-intensity kernels hit the sloped memory ceiling, while high-intensity ones hit the flat compute ceiling.

Fixing the wrong wall wastes effort. Adding math to a memory-bound kernel changes nothing; you must cut data traffic instead.

To speed a memory-bound kernel, coalesce accesses, reuse data in shared memory, and cache values to read less.

For compute-bound work, raise parallelism, use faster math, or reach for tensor cores to push past the math ceiling.

In practice, the majority of CUDA kernels are memory-bound. Bandwidth, not arithmetic, is usually the scarce resource.

Do not guess the wall. The roofline in Nsight Compute places your kernel under the correct ceiling for you.

Ask one question: are the cores or the memory pipes closer to peak? Whichever is saturated names your bound.

A kernel has very low arithmetic intensity.

Diagnose the wall first: memory-bound kernels need less traffic, compute-bound ones need more math. The roofline tells you which. 👏