Prefetching with cudaMemPrefetchAsync
Moving pages before they are needed.
Prefetching with cudaMemPrefetchAsync is a free CUDA Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the CUDA Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Stop Paying for Faults
Instead of waiting for slow first-touch faults, you can move pages early. cudaMemPrefetchAsync sends managed data to a device before the kernel needs it.
The Basic Call
You name the pointer, the byte count, and the destination device. This one prefetch migrates the whole range up front in a single efficient move.
cudaMemPrefetchAsync(data, n * sizeof(float), 0);Pick the Destination
The third argument is the device id. Pass a GPU number to stage data on that GPU, ready for the kernel you are about to launch.
int dev = 0;
cudaMemPrefetchAsync(data, bytes, dev);Prefetch Back to the CPU
Use the special id cudaCpuDeviceId to pull results back to the host. Do it before CPU code reads them to avoid a wave of faults.
cudaMemPrefetchAsync(data, bytes, cudaCpuDeviceId);It Is Asynchronous
The Async in the name is real: the call returns immediately and runs in a stream. Your CPU keeps working while pages migrate in the background.
Overlap With Compute
Because it rides a stream, a prefetch can overlap with other kernels. Stage the next chunk while the current one is still being processed.
cudaMemPrefetchAsync(next, bytes, dev, stream);One Move Beats Many Faults
A single bulk prefetch is far cheaper than thousands of tiny faults. You trade scattered overhead for one contiguous high-bandwidth transfer.
Prefetch the Right Range
Only stage what the kernel actually touches. Prefetching a huge buffer the kernel barely reads just wastes bandwidth and GPU memory.
A Two-Sided Pattern
A clean rhythm emerges: prefetch to the GPU, launch the kernel, prefetch results back. This keeps migration off the critical path on both ends.
Measure, Do Not Guess
Add a prefetch, then check Nsight for fewer faults and tighter timelines. Let profiling confirm the win rather than trusting intuition.
Convenience Plus Control
Prefetching keeps the single-pointer ease of managed memory while giving you back control over timing. You get the best of both styles.
Quick Check
Let us confirm what prefetching buys you.
Recap: Prefetching
You learned to stage pages early with cudaMemPrefetchAsync, picking a GPU or cudaCpuDeviceId. It overlaps in streams and beats faulting. Great job! ✨
Frequently asked questions
Is the “Prefetching with cudaMemPrefetchAsync” lesson free?
Yes — the full text of “Prefetching with cudaMemPrefetchAsync” is free to read here on the web, and the CUDA Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the CUDA Academy course, upgrade to CoddyKit PRO.
What will I learn in “Prefetching with cudaMemPrefetchAsync”?
Moving pages before they are needed. You practise CUDA Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start CUDA Academy?
No prior experience is required. CUDA Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Prefetching with cudaMemPrefetchAsync” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this CUDA Academy lesson?
Yes. Every CUDA Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- One Pointer, Both Sides
- On-Demand Page Migration
- Prefetching with cudaMemPrefetchAsync
- Hints via cudaMemAdvise