Backprop a Tiny Net by Hand
Compute gradients on paper, confirm in code.
Backprop a Tiny Net by Hand is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Our Tiny Network
Let's hand-trace the smallest net: one input x, one weight w, no bias. The output is simply y = w times x. Tiny enough to do on paper.
y = w * xAdd a Simple Loss
We compare the output to a target t with squared error. This loss punishes being far from the target and is easy to differentiate by hand.
loss = (y - t) ** 2Forward With Numbers
Plug in x = 2, w = 3, t = 10. Forward gives y = 6 and loss = (6 - 10) squared = 16. Now we walk backward to find dloss/dw.
Step One: Loss to Output
First link: how does loss change with y? The derivative of (y - t) squared is 2(y - t), which here is 2 times (6 - 10) = -8.
dloss_dy = 2 * (y - t)Step Two: Output to Weight
Second link: how does y change with w? Since y = w times x, dy/dw is just x, which is 2 in our example.
dy_dw = xChain the Two Together
The chain rule multiplies the links: dloss/dw = dloss/dy times dy/dw = -8 times 2 = -16. That single number is our gradient.
dloss_dw = dloss_dy * dy_dwRead the Gradient's Sign
A negative gradient means increasing w would lower the loss. So we should nudge w upward to do better next time.
Take One Update Step
With a learning rate of 0.1, the update is w = w - 0.1 times (-16) = 3 + 1.6 = 4.6. The weight moved toward a better value.
w = w - lr * dloss_dwConfirm It Improved
Redo the forward pass with w = 4.6: y = 9.2 and loss = (9.2 - 10) squared = 0.64. Far below 16, so the step truly helped.
Check It in PyTorch
PyTorch gets the same gradient automatically. Set requires_grad on w, run forward, call backward, and read w.grad to see -16.
w = torch.tensor(3.0, requires_grad=True)
loss = (w * 2 - 10) ** 2
loss.backward()
print(w.grad)Same Steps, Bigger Nets
A million-weight net does exactly this, just with more links chained together. The math you did by hand scales straight up.
Quick Check
Let's check your hand trace.
Recap
You traced a tiny net: forward for values, then backward multiplying local derivatives to get the gradient, then one step that lowered the loss. ✏️
Frequently asked questions
Is the “Backprop a Tiny Net by Hand” lesson free?
Yes — the full text of “Backprop a Tiny Net by Hand” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Backprop a Tiny Net by Hand”?
Compute gradients on paper, confirm in code. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Backprop a Tiny Net by Hand” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The Chain Rule, Layer by Layer
- Forward Caches, Backward Reuses
- Backprop a Tiny Net by Hand
- Vanishing & Exploding Gradients