Call backward() to Get Gradients
Fill .grad with one line.
Call backward() to Get Gradients is a free Deep Learning Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
One Line to Differentiate
Once your graph is built, a single call does all the calculus. Run backward() on your output and gradients flow back to every input.
Start From a Scalar
You call backward on the final number you want to minimize, usually the loss. It must be a single scalar so autograd knows where to start.
y = x ** 2
y.backward()Gradients Land in .grad
After backward runs, each leaf tensor's .grad holds the derivative of the output with respect to that tensor. The math is already done.
print(x.grad)A Quick Check by Hand
For y equal to x squared, calculus says the derivative is 2x. At x equal to 2 that is 4, and PyTorch fills x.grad with exactly 4.0.
Backward Walks the Graph
backward() travels the graph from output to inputs, applying the chain rule at every node. You wrote only the forward math and got all of this free.
Gradients for Many Inputs
If your output depends on several tracked tensors, one backward call fills the .grad of each of them at once. That is how whole models update.
loss = (a * b + c).sum()
loss.backward()The Graph Is Spent
By default the graph is freed after backward to save memory, so calling it twice errors. You build a fresh graph on the next forward pass.
Keep It If You Must
Need to call backward again on the same graph? Pass retain_graph equals true. You rarely need this, so skip it unless an error tells you to.
loss.backward(retain_graph=True)Non-Scalar Outputs
If your output is a vector, backward needs a gradient argument saying how to weight each element. Most of the time you sum to a scalar instead.
This Is the Backward Pass
That single backward call is the famous backward pass of training. Forward computes predictions, backward computes how to fix the weights.
From Gradients to Learning
The numbers in .grad tell each weight which way to move. An optimizer reads them and nudges the weights to lower the loss a little.
Quick Check
Recap
Call backward() on a scalar loss and autograd applies the chain rule across the graph, filling each tensor's .grad with its derivative. That is the backward pass. 🔁
Frequently asked questions
Is the “Call backward() to Get Gradients” lesson free?
Yes — the full text of “Call backward() to Get Gradients” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Call backward() to Get Gradients”?
Fill .grad with one line. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Call backward() to Get Gradients” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- requires_grad and the Computation Graph
- Call backward() to Get Gradients
- Reading and Zeroing .grad
- torch.no_grad() for Inference