การแบ่งบล็อกเพื่อความใกล้เคียงของแคช

The CPU keeps recently used data in a small, fast cache. Hits are quick; misses force a slow trip to main memory.

Cache locality means reusing data that is already nearby in the cache before it gets evicted to make room.

If your kernel sweeps a huge array, early elements are evicted before you reuse them, causing repeated cache misses.

Tiling splits a big loop into small blocks that fit in cache, so each block's data stays hot while you use it.

Tiling turns one loop into two: an outer loop over blocks and an inner loop over the elements inside each block.

Each block is small enough to live in cache. You finish all its work before moving on, so reuse stays cheap.

Matrix multiply reuses rows and columns heavily. Blocking a matmul into tiles keeps reused data in cache and cuts misses.

The best tile fills the cache without overflowing it. Too small wastes reuse; too large spills, so you tune the size.

Tiling and SIMD stack well. Vectorize the inner loop of each block to get cache locality and vector speed together.

The last block may be smaller than a full tile. Clamp its range with min so you never read past the buffer.

Tiling can help a lot or a little depending on sizes. Always benchmark a few tile values on your real data to choose.

Your kernel keeps missing cache because it sweeps a huge array end to end. What technique helps?

Tiling blocks a big loop so each chunk fits in cache; tune the tile size, vectorize the inner loop, and clamp the edges. 🧱