Matrix Multiply with matmul and @
The dot products behind every layer.
Beyond Elementwise
Multiplying tensors with the star sign is elementwise. Matrix multiplication is different: it mixes rows and columns into weighted sums.
Dot Products in Bulk
Each output entry of a matrix multiply is a dot product: one row of the left matrix paired with one column of the right matrix.
All lessons in this course
- Why Loops Are Slow for Math
- Elementwise Ops & Reductions
- Matrix Multiply with matmul and @
- Dot Products Power Every Layer