A Mental Model of the Hierarchy
Matching data to the right space.
A Pyramid of Tradeoffs
Think of GPU memory as a pyramid. The top is tiny and lightning fast, while the base is huge but slow. Your job is to use each layer well.
Top: Registers
At the peak sit registers: per-thread, fastest, and very limited. Keep your hottest working values here whenever you possibly can.
All lessons in this course
- Registers and Local Memory
- Global Memory Tradeoffs
- Constant Memory and Its Cache
- A Mental Model of the Hierarchy