LSTM과 GRU 게이트

Vanilla RNNs forget early clues over long sequences. Gated cells fix this by deciding what to keep, update, or throw away.

The LSTM adds a separate cell state, a memory highway that runs straight across time with only small, controlled edits.

A gate is a sigmoid layer outputting values from 0 to 1. Zero blocks information, one lets it pass, and in between mixes the two.

The forget gate looks at the input and memory and chooses which parts of the old cell state to erase before adding anything new.

The input gate decides how much of the fresh candidate information should be written into the cell state at this step.

The output gate controls how much of the updated cell state becomes the visible hidden state passed to the next step. 🚪

Because the cell state changes gently, gradients flow back across many steps without vanishing, so the model learns long-range patterns.

The GRU is a lighter cousin: it merges gates and drops the separate cell state, giving similar power with fewer parameters.

A GRU uses just two gates, a reset gate and an update gate, to balance old memory against new input each step.

Both are one-liners in PyTorch. Swap nn.LSTM or nn.GRU for nn.RNN and keep almost the same training code.

GRUs are faster and often match LSTMs; LSTMs can edge ahead on the hardest long sequences. Try both and let your validation score decide.

What is the job of the forget gate in an LSTM?

LSTMs and GRUs use gates to control memory, letting gradients survive long sequences and capturing far-apart dependencies. ✅