Optimizing the Inner Product
Vectorize and unroll the dot product.
Optimizing the Inner Product is a free Mojo Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Mojo Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Dot Product Core
The inner product multiplies two sequences element by element and adds the results. Speeding it up speeds your whole matmul.
var acc: Float32 = 0.0
for k in range(K):
acc += a[k] * b[k]One Lane at a Time Is Slow
A plain loop handles a single pair per step. Modern CPUs can do many at once, so scalar code leaves lanes idle.
Enter SIMD
A SIMD value packs several numbers and applies one operation to all of them together, doing real work in parallel per instruction.
var v = SIMD[DType.float32, 4](1, 2, 3, 4)Load a Chunk at Once
Instead of one float, load a whole SIMD-width slice of each array in a single move, ready for vector math.
var va = a.load[width=4](k)
var vb = b.load[width=4](k)Multiply Whole Vectors
Multiplying two SIMD values yields a vector of products in one step. The element-wise work happens across all lanes together.
var prod = va * vb # 4 products at onceAccumulate Into a Vector
Keep a SIMD accumulator and add each product vector into it. You delay the final sum until the loop is done.
var acc = SIMD[DType.float32, 4](0)
acc += va * vbStep by the Width
The loop now advances by the SIMD width, not by one. Four lanes wide means a quarter as many iterations.
for k in range(0, K, 4):
acc += a.load[width=4](k) * b.load[width=4](k)Reduce at the End
After the loop, collapse the lane accumulator into one number with a horizontal reduce, giving the final dot product.
var total = acc.reduce_add()Unrolling the Loop
Unrolling processes several widths per iteration, cutting loop overhead and exposing more independent work to the CPU.
Handle the Remainder
If K is not a multiple of the width, a few elements are left over. A small tail loop finishes them one at a time.
for k in range(K - K % 4, K):
total += a[k] * b[k]Let Mojo Help
Mojo offers the vectorize helper to apply a width-parameterized body across a range, handling stepping and the tail for you.
Quick Check
After a SIMD-width dot product loop, why do you call reduce_add at the end?
Recap
Vectorize the inner product by loading SIMD chunks, multiplying lanes together, accumulating, then reduce; unroll and clean up the tail. ⚡
Frequently asked questions
Is the “Optimizing the Inner Product” lesson free?
Yes — the full text of “Optimizing the Inner Product” is free to read here on the web, and the Mojo Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Mojo Academy course, upgrade to CoddyKit PRO.
What will I learn in “Optimizing the Inner Product”?
Vectorize and unroll the dot product. You practise Mojo Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Mojo Academy?
No prior experience is required. Mojo Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Optimizing the Inner Product” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Mojo Academy lesson?
Yes. Every Mojo Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Modeling a Tensor in Mojo
- Building a Matmul Step by Step
- Optimizing the Inner Product
- Verifying Numeric Correctness