การส่งต่อเก็บแคช การย้อนกลับนำกลับมาใช้

Training each batch runs two passes: a forward pass to predict and compute loss, then a backward pass to compute gradients. They work as a pair.

Going forward, each layer turns its input into an output and sends it onward. By the end you have a prediction and a single loss number.

To compute a layer's gradient, the chain rule needs the very activations that layer produced going forward. Those values are not optional.

During the forward pass the network quietly caches each layer's inputs and outputs in memory, ready for the backward pass to grab. 💾

The derivative of a layer often reuses its own output. For a sigmoid, the gradient depends on the saved output value, so caching it saves recomputation.

The backward pass walks layers in reverse, and at each one it pulls the matching cached values to multiply into the gradient. Nothing is recomputed.

Storing every activation is why training a deep net uses far more memory than just running it for predictions. Bigger nets need bigger caches.

When you only need predictions, there is no backward pass, so PyTorch skips the cache entirely. That is why inference is lighter on memory.

Every operation on a tensor with requires_grad records what it needs into the computation graph, building the cache automatically as you go.

By default, calling backward() consumes the cached graph and frees it. That is why a second backward() on the same graph raises an error.

Caching forward values means each gradient is one cheap lookup-and-multiply instead of a fresh recomputation, making backprop fast and exact.

Let's check the cache idea.

The forward pass caches activations, and the backward pass reuses them to build gradients. That trade of memory for speed is what makes backprop practical. 💾