Ce que calculent les cœurs tensoriels

A Tensor Core is a special hardware unit on modern NVIDIA GPUs built to do one job blazingly fast: small matrix math. 🚀

Tensor Cores compute a matrix multiply-accumulate, written D = A times B plus C. They do the whole multiply-and-add in one shot.

Instead of one number, a Tensor Core handles a small matrix tile each cycle. That is why it crushes the work a plain core does element by element.

The multiply and the add happen fused, so the intermediate product is never rounded on its own. That keeps more accuracy than two separate steps.

Deep learning and graphics are full of matrix multiplies. Tensor Cores exist because that one operation dominates so many real workloads.

The plus C in D = A times B plus C lets you keep a running total. This accumulator is how big results are built from many small tiles.

For matrix math a Tensor Core can deliver many times the throughput of ordinary CUDA cores, because it packs a full tile multiply into one instruction.

Tensor Cores often take low-precision inputs but keep a higher-precision accumulator. You get speed on the multiply and safety on the running sum.

Tensor Cores arrived with the Volta architecture and grew with Turing, Ampere, and Hopper. Each generation widened the tiles and added formats.

Tensor Cores only help when your work looks like a matrix multiply. A simple vector add or scalar loop will not touch them at all.

You reach Tensor Cores through libraries or the WMMA API. This call shape is the multiply-accumulate you will write later.

What single operation are Tensor Cores designed to accelerate?

You learned that a Tensor Core fuses a full tile-sized matrix multiply-accumulate into one fast step, perfect for the matrix math behind deep learning. Next: precision formats. 🎉