requires_grad and the Computation Graph
Track operations to differentiate them.
requires_grad and the Computation Graph is a free Deep Learning Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Calculus, Handled For You
Training needs derivatives, but you never compute them by hand. PyTorch's autograd watches your math and works out every gradient for you. 🤖
Tensors Can Track Themselves
A tensor only earns gradients when you ask. Set requires_grad to true and PyTorch starts recording every operation done to it.
x = torch.tensor(2.0, requires_grad=True)What Gets Recorded
Each math step on a tracked tensor is logged as a node. Together these nodes form a computation graph describing how your result was built.
The Graph Is a Recipe
Think of the graph as a recipe: inputs at the top, operations in the middle, your final loss at the bottom. Autograd reads it backward to find gradients.
Built On the Fly
PyTorch uses a dynamic graph: it is created as your code runs, not ahead of time. Normal Python loops and ifs just work inside it.
Results Stay Connected
Any tensor made from a tracked one is also tracked. Here y remembers it came from x, so its grad_fn points back to that squaring step.
y = x ** 2
print(y.grad_fn)Leaves vs Computed Nodes
Tensors you create directly are leaf nodes. Tensors produced by operations are interior nodes that link back toward those leaves.
grad_fn Names the Step
Every computed tensor carries a grad_fn telling autograd which operation made it, like PowBackward or AddBackward. Leaves have no grad_fn.
No Tracking, No Graph
If a tensor has requires_grad false, autograd ignores it and builds no graph. That saves memory whenever you do not need gradients.
Why the Graph Matters
The graph is what makes the backward pass possible. Without this recorded history, PyTorch would have no way to know how the loss depends on each weight.
You Just Write Forward
The beauty is you only code the forward math. Autograd silently assembles the graph so the gradients are ready the moment you ask for them.
Quick Check
Recap
Set requires_grad and PyTorch records your math into a computation graph. That recorded history is what autograd later reads backward to find every gradient. ✨
Frequently asked questions
Is the “requires_grad and the Computation Graph” lesson free?
Yes — the full text of “requires_grad and the Computation Graph” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “requires_grad and the Computation Graph”?
Track operations to differentiate them. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “requires_grad and the Computation Graph” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- requires_grad and the Computation Graph
- Call backward() to Get Gradients
- Reading and Zeroing .grad
- torch.no_grad() for Inference