Die Thread-Hierarchie

A CUDA kernel runs in parallel across many threads, and each thread does the same work on its own slice of the data. 🧵

Launching millions of loose threads would be chaos. CUDA organizes them into a clean hierarchy so the hardware can schedule them well.

Threads are grouped into a block. Every thread in a block runs the same kernel and can cooperate closely with its neighbors.

All the blocks for one launch together make up the grid. So the chain is simple: threads live in blocks, and blocks live in the grid.

The two-level design lets one kernel scale from a tiny GPU to a huge one. You describe the work once and the grid stretches to fit.

You choose how many blocks and how many threads per block right at the launch, using the triple-angle-bracket syntax.

Say you launch 4 blocks of 256 threads. That gives you 4 times 256, or 1024 threads all running at once.

Different blocks cannot assume any order or talk to each other directly. This independence is exactly what lets the GPU run them in any sequence.

Inside one block, threads can share fast on-chip memory and synchronize with each other. That teamwork is a block's superpower.

Both the grid and each block can be 1D, 2D, or 3D. The dim3 type lets you map naturally onto images or volumes.

The common goal is to give each thread exactly one data element to handle. The hierarchy is just how you reach that mapping.

Let us check how the pieces nest together.

You learned the CUDA ladder: threads make a block, blocks make the grid. Independent blocks scale, while threads inside a block cooperate. 🎯