La vie d’un programme CUDA

Almost every CUDA program follows the same five-beat dance. Learn the rhythm once and you can read any GPU code. 🕺

First reserve device memory for your inputs and outputs with cudaMalloc, since the GPU cannot use host buffers directly.

Next upload your input data from host to device with cudaMemcpy. The GPU now has its own copy to work on.

Now launch your kernel across many threads. This is where the parallel work actually happens on the device. 🚀

When compute finishes, download the results from device to host so the CPU can read and use them.

Finally release every device buffer with cudaFree. Skipping this leaks GPU memory that other work could use.

Because launches are async, call cudaDeviceSynchronize before reading results to ensure the GPU has truly finished.

The copy steps cross the slow PCIe bus, so data transfer is often the real bottleneck, not the kernel itself.

Allocating once and reusing buffers across many launches beats allocating fresh memory every iteration of a loop.

Notice the symmetry: every cudaMalloc pairs with a cudaFree, and every copy in eventually pairs with a copy out.

Say it like a mantra: allocate, copy, launch, copy back, free. That single line is the skeleton of nearly every kernel program.

Let us check the standard CUDA program lifecycle.

You learned the five-step CUDA lifecycle: allocate, copy in, launch, copy back, free, plus syncing before you read results. 🎉