อ่านและล้างค่า .grad

After backward, every trained tensor stores its gradient in .grad. Reading it tells you how the loss responds to changes in that tensor.

Before any backward call, .grad is None, not zero. PyTorch only allocates it once the first set of gradients actually arrives.

Here is the surprise: backward adds to .grad rather than replacing it. Call backward twice without clearing and the numbers pile up.

This adding behavior is on purpose. It lets you sum gradients from several mini-batches before one update, handy for simulating a larger batch.

Forget to clear and old gradients poison your next step, so the model trains on the wrong numbers. This silent bug trips up almost everyone once.

The fix is to reset gradients before each backward. With an optimizer you simply call zero_grad() at the top of every training step.

The loop rhythm is fixed: zero_grad, forward, loss, backward, step. Zeroing first guarantees each step uses only this batch's gradients.

No optimizer yet? You can null the gradients yourself by setting each tensor's .grad back to None before the next backward call.

Newer code prefers zero_grad(set_to_none=True). Setting grads to None instead of filling zeros saves a little memory and time.

Peeking at .grad is great for debugging. All zeros may mean a dead neuron, and huge values warn of exploding gradients before training blows up.

Make zeroing automatic in your head. Every reliable training loop clears gradients first, so the model only ever learns from the current batch.

Gradients live in .grad and accumulate across backward calls, so you must clear them each step with zero_grad. Forgetting that quietly breaks training. 🧹