Forward Pass, Loss, Backward, Step
The four moves of one training iteration.
Forward Pass, Loss, Backward, Step is a free Deep Learning Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Four Moves, One Beat
Every training iteration is just four moves repeated: forward, loss, backward, step. Learn this rhythm and the whole training loop stops feeling mysterious. 🥁
Move One: Forward
The forward pass sends your input through the model so it produces a prediction. You just call the model on a batch of data and read the output.
pred = model(x)Logits, Not Answers Yet
The forward output is usually raw scores called logits, not clean labels. The loss function knows how to read these scores directly, so you leave them as is.
Move Two: Loss
Next you measure how wrong the prediction was. The loss compares your output to the true labels and boils the mistake down to one number.
loss = loss_fn(pred, y)Lower Loss, Better Model
A big loss means a poor prediction and a small one means a good prediction. Training has a single mission: drive this number steadily downward.
Move Three: Backward
Now you ask which way to nudge each weight. The backward pass uses autograd to compute every gradient and fill each parameter's .grad for you.
loss.backward()Gradients Are Directions
Each gradient tells you how the loss would change if you tweaked that one weight. They are the map the optimizer follows toward a better model.
Move Four: Step
Finally the optimizer applies those gradients, nudging every weight a little. This step is the moment the model actually learns from its mistake.
optimizer.step()Clear the Grads
Gradients pile up by default, so before the next backward you call zero_grad. Skip it and old gradients corrupt your next update.
optimizer.zero_grad()The Whole Move
Put the four moves together and you get one clean iteration. You will type this exact pattern in nearly every PyTorch project you build.
optimizer.zero_grad()
pred = model(x)
loss = loss_fn(pred, y)
loss.backward()
optimizer.step()Then Do It Again
Run these moves on batch after batch and the loss falls a little each time. Thousands of tiny updates are what turn random weights into real skill.
Quick Check
Which call actually computes the gradients for every weight?
Recap
One iteration is four moves: forward for a prediction, loss to measure error, backward for gradients, and step to update weights. Repeat to learn. 🎉
Frequently asked questions
Is the “Forward Pass, Loss, Backward, Step” lesson free?
Yes — the full text of “Forward Pass, Loss, Backward, Step” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “Forward Pass, Loss, Backward, Step”?
The four moves of one training iteration. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Forward Pass, Loss, Backward, Step” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Forward Pass, Loss, Backward, Step
- Write a Minimal Training Loop
- Track Accuracy While You Train
- Train vs Eval Mode