페이지 가능 메모리가 느린 이유

Normal C++ buffers from malloc sit in pageable host memory. It feels free, but the GPU cannot copy from it directly, and that small detail costs you speed.

Memory is pageable when the operating system may move its pages around or swap them to disk at any moment. Great for flexibility, tricky for hardware that needs fixed addresses.

The GPU pulls data using DMA, a direct hardware transfer that needs a physical address that will not move. Pageable pages break that promise, so they cannot be used directly.

To work around this, the driver copies your pageable buffer into a hidden staging buffer first, then sends that to the GPU. You pay for an extra copy you never wrote. 😮

So a single cudaMemcpy from pageable memory is really two copies: host to staging, then staging to device. That doubled work is exactly why pageable transfers feel sluggish.

Here is the buffer everyone reaches for first. It works, but every transfer from h_data quietly goes through that staging detour.

The OS keeps memory pageable so it can overcommit RAM and serve many programs at once. That convenience is what blocks the GPU from reading your pages directly.

Pageable transfers often reach only half of your link's peak bandwidth. The staging copy eats CPU cycles and memory traffic that real transfers could have used.

Because the staging copy is synchronous, pageable transfers cannot truly overlap with kernels. You lose the async tricks that make streams worthwhile.

For one tiny copy nobody notices. But in a loop that ships data every iteration, the staging tax compounds and quietly dominates your runtime.

The cure is asking CUDA for a buffer that cannot be paged out, called pinned memory. The GPU reads it directly, no staging, no doubled copy.

Why is a transfer from ordinary pageable memory slower than it needs to be?

Pageable buffers can move, so the GPU cannot DMA them directly. The driver stages them first, doubling the copy. The fix ahead is pinned memory. 🚀