Sequenzielle Adressierung

Shared memory is split into 32 banks, one per warp lane. When 32 threads hit 32 different banks, all reads happen in a single fast cycle.

If two threads in a warp touch the same bank, that is a bank conflict. The hardware serializes those accesses, costing extra cycles.

The previous reduction used interleaved addressing: stride starts small and doubles, so partners are close together in shared memory.

With small, doubling strides, several lanes in a warp map onto the same bank. Those accesses can no longer happen in one cycle.

Sequential addressing starts the stride large and halves it each step, the reverse of interleaving. This single change removes the conflicts.

A large stride spreads partner addresses far apart, so each lane lands on its own bank. The warp reads conflict-free in one cycle.

Only the lower half of threads work each step, written as tid < s. That keeps active threads contiguous, so warps stay non-divergent too.

Sequential addressing fixes bank conflicts and avoids warp divergence in the same kernel. One layout change, two performance problems solved.

You still need a __syncthreads after each step. Threads must see the previous level's writes before they read for the next level.

As the stride halves toward zero, all partial sums fold into data[0]. Thread 0 then writes that block's result back to global memory.

This pattern comes straight from NVIDIA's famous reduction guide. Sequential addressing is a textbook step toward a conflict-free kernel.

Think about why a large, halving stride beats a small, doubling one.

You swapped interleaved for sequential addressing: stride starts large and halves, killing bank conflicts and divergence at once. Up next: multi-block sums. ✨