Race Conditions auf der GPU

On the GPU, thousands of threads can touch the same memory at once. When several write to one spot together, you get a race condition. 🏁

A race condition happens when the final result depends on the unpredictable order in which threads run. Same code, different answer each launch.

Incrementing a counter is really three steps: read the value, add one, write it back. This read-modify-write sequence is where things break.

Two threads both read 5, both compute 6, both write 6. You expected 7 but got 6. One increment silently vanished.

That vanished increment is called a lost update. With thousands of threads racing, dozens or hundreds of updates can disappear at once.

This kernel looks correct, but every thread races on the same address. The final count will be wrong and will vary between runs.

The hardware never promises a thread order. So a racy kernel is nondeterministic: it may even pass on small inputs and fail on big ones.

Many threads reading the same value is perfectly fine. Trouble starts only when at least one thread writes while others read or write.

If each thread writes its own unique slot, like out[i], there is no conflict. A race needs threads aiming at the same location.

The cure is to make read-modify-write happen as one indivisible step. That is an atomic operation, coming up in the next lesson. ⚛️

When reviewing a kernel, ask: do two threads write the same address without protection? If yes, you almost certainly have a data race.

Let's make sure the race idea clicked.

You learned that unguarded read-modify-write on shared data causes race conditions and lost updates. Atomics, up next, make those steps indivisible. ✅