Dropout: 뉴런을 무작위로 제외하기

Dropout randomly turns off some neurons during each training step, so the network cannot lean on any single unit too heavily. 🎲

By dropping neurons at random, dropout forces the network to build redundant paths, which spreads knowledge and improves generalization.

The dropout rate p is the chance each neuron is dropped. A value like 0.5 means roughly half the units are silenced each step.

You add dropout with one layer. Place nn.Dropout between linear layers to regularize the activations flowing through.

Each forward pass drops a different random set of neurons, so the model effectively trains a huge ensemble of thinner networks.

To keep the average signal steady, PyTorch scales the surviving activations up during training, so test-time outputs stay balanced.

Dropout must be disabled during evaluation. Calling model.eval() switches it off so every neuron contributes to the prediction.

Common rates sit between 0.2 and 0.5. Higher values regularize more but can starve the network of signal if pushed too far.

Put dropout on wide hidden layers where overfitting bites hardest. It is rarely applied right before the final output.

For convolutional features, plain dropout helps less. Many vision nets prefer Dropout2d or rely on batch norm instead.

Dropout adds no extra parameters and costs almost nothing, yet it is one of the most reliable ways to shrink the train/val gap.

You met dropout: randomly silencing neurons in training to force redundancy, then turning it off at eval for full, stable predictions. ✅