Broadcasting Rules That Save You Loops
Combine different-shaped tensors elementwise.
Broadcasting Rules That Save You Loops is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Combine Without Matching Shapes
Broadcasting lets PyTorch stretch a smaller tensor to fit a bigger one, so you skip writing loops to repeat values.
Add a Scalar to Everything
The simplest broadcast: add one number to a whole tensor. PyTorch applies it to every element at once. ✨
x = torch.tensor([1, 2, 3])
print(x + 10) # tensor([11, 12, 13])Compare Shapes Right to Left
Broadcasting lines up dimensions from the right. It then checks each pair to decide if they can combine.
Rule One: Equal Sizes Match
Two dimensions are compatible when they are equal. Matching sizes line up one to one with no stretching needed.
Rule Two: A Size of 1 Stretches
If one dimension is 1, it expands to match the other. That single value is reused across the whole axis.
a = torch.tensor([[1], [2], [3]])
b = torch.tensor([10, 20])
print((a + b).shape) # torch.Size([3, 2])Missing Dimensions Count as 1
When one tensor has fewer dimensions, PyTorch pads it with leading ones. A vector can broadcast against a matrix this way.
m = torch.ones(2, 3)
v = torch.tensor([1, 2, 3])
print((m + v).shape) # torch.Size([2, 3])Add a Bias Across Rows
A classic use: add a row bias to every sample in a batch. One small vector reaches every row for free.
batch = torch.zeros(4, 3)
bias = torch.tensor([1.0, 2.0, 3.0])
print((batch + bias).shape) # torch.Size([4, 3])Normalize a Column at Once
Subtract a per-column mean with broadcasting and center your data in one line, no loop over rows required.
x = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
m = x.mean(dim=0)
print(x - m)When Shapes Don't Broadcast
If two dimensions differ and neither is 1, the broadcast fails and PyTorch raises a clear size error.
Add Dimensions to Steer Broadcasting
Use unsqueeze to insert a size-1 axis exactly where you need it. This guides broadcasting toward the shape you want.
col = torch.tensor([1, 2, 3]).unsqueeze(1)
print(col.shape) # torch.Size([3, 1])Why Broadcasting Is Fast
Broadcasting never copies the stretched values, it reuses them. That makes vectorized math far faster than Python loops. 🚀
Quick Check
Can you predict whether two shapes will broadcast?
Recap: Broadcasting Rules
You learned the two rules: dimensions match when equal or when one is 1. Master this and loops melt into clean, fast tensor math. 💪
Frequently asked questions
Is the “Broadcasting Rules That Save You Loops” lesson free?
Yes — the full text of “Broadcasting Rules That Save You Loops” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Broadcasting Rules That Save You Loops”?
Combine different-shaped tensors elementwise. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Broadcasting Rules That Save You Loops” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Shapes, Dtypes & Indexing
- Reshape, View, Squeeze & Unsqueeze
- Broadcasting Rules That Save You Loops
- Tensors Talk to NumPy