تقليل استخدام ذاكرة GPU

Run out of GPU memory and training crashes with an out-of-memory error. The good news is several simple tactics free up space fast.

Your GPU holds the model weights, the gradients, the optimizer state, and the activations saved for backward. Activations are often the biggest.

The fastest fix is a smaller batch size. Fewer samples per step means fewer activations to store, and accumulation can recover the effective size.

During evaluation you do not need gradients. Wrapping inference in torch.no_grad skips storing activations and saves a lot of memory.

Half-precision tensors are simply smaller. Turning on autocast cuts activation and weight memory roughly in half during training.

Checkpointing trades compute for memory: it drops most activations and recomputes them during backward instead of keeping them all.

Wrap a heavy block in checkpoint so its activations are rebuilt on the backward pass. You save memory at the cost of extra recompute.

Keeping a loss tensor around holds its whole graph in memory. Call .item() to log just the number and let the graph be freed.

Pass set_to_none to zero_grad so gradient tensors are released instead of merely filled with zeros, freeing their memory between steps.

PyTorch caches freed blocks for reuse. When you truly need space back, empty_cache returns it to the GPU, though it rarely fixes real leaks.

Check how much you hold with memory_allocated. Watching this number while you tune confirms which change actually freed space.

Which technique saves memory by recomputing activations during backward?

Beat out-of-memory by shrinking batches, using no_grad for inference, autocast, gradient checkpointing, and set_to_none. Measure with memory_allocated. 🧹