Elementwise Ops & Reductions
Sum, mean, max across chosen axes.
Elementwise Ops & Reductions is a free Deep Learning Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Two Kinds of Operations
Tensor math splits into two families: elementwise ops that keep the shape, and reductions that collapse it down to fewer numbers.
Elementwise Keeps the Shape
An elementwise op applies the same action to every entry independently. Input shape in, same shape out, no mixing between positions.
b = a * 2 + 1
# same shape as a, every element transformedPairwise Elementwise Math
With two tensors of equal shape, elementwise ops act position by position. Add matches index to index, multiply does the same.
c = a + b
d = a * bMath Functions Are Elementwise Too
Functions like torch.relu, exp, and sqrt run elementwise. Each number is transformed on its own, and the tensor keeps its original shape.
r = torch.relu(x)
e = torch.exp(x)Reductions Collapse Numbers
A reduction combines many values into fewer. Sum, mean, and max fold a whole tensor down, by default to a single scalar.
total = x.sum()
avg = x.mean()The dim Argument Picks an Axis
Pass dim to reduce along one axis only. The chosen axis disappears while the others stay, so a 2D tensor becomes 1D.
col_sums = x.sum(dim=0)
row_means = x.mean(dim=1)keepdim Saves the Shape
Set keepdim=True to keep the reduced axis as size 1. That preserved shape is what makes later broadcasting line up cleanly.
m = x.max(dim=1, keepdim=True).valuesMean Needs Floats
mean divides, so it expects floating-point input. Call it on an integer tensor and PyTorch will complain until you cast to float first.
avg = x.float().mean()argmax Finds the Winner
Sometimes you want the position, not the value. argmax returns the index of the largest entry, which is how a classifier picks its predicted class.
pred = logits.argmax(dim=1)Chain Them Together
Real code stacks both kinds: an elementwise transform feeds a reduction. Square the errors, then take the mean, and you have mean squared error.
mse = ((pred - target) ** 2).mean()Pick Shape-Keeping or Shape-Shrinking
The rule of thumb: reach for elementwise when every value should change in place, and reach for a reduction when you need a summary like a total or average.
Quick Check
Can you tell which operation changes a tensor shape?
Recap: Transform vs Summarize
Elementwise ops keep the shape and act per value; reductions like sum and mean collapse it, with dim and keepdim controlling exactly how. 🎯
Frequently asked questions
Is the “Elementwise Ops & Reductions” lesson free?
Yes — the full text of “Elementwise Ops & Reductions” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Elementwise Ops & Reductions”?
Sum, mean, max across chosen axes. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Elementwise Ops & Reductions” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Loops Are Slow for Math
- Elementwise Ops & Reductions
- Matrix Multiply with matmul and @
- Dot Products Power Every Layer