LSTM & GRU Gates
Remember long-range dependencies.
LSTM & GRU Gates is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Long-Memory Problem
Vanilla RNNs forget early clues over long sequences. Gated cells fix this by deciding what to keep, update, or throw away.
Meet the LSTM
The LSTM adds a separate cell state, a memory highway that runs straight across time with only small, controlled edits.
Gates Are Soft Switches
A gate is a sigmoid layer outputting values from 0 to 1. Zero blocks information, one lets it pass, and in between mixes the two.
gate = torch.sigmoid(W @ x + U @ h)The Forget Gate
The forget gate looks at the input and memory and chooses which parts of the old cell state to erase before adding anything new.
The Input Gate
The input gate decides how much of the fresh candidate information should be written into the cell state at this step.
The Output Gate
The output gate controls how much of the updated cell state becomes the visible hidden state passed to the next step. 🚪
Why Gates Help Gradients
Because the cell state changes gently, gradients flow back across many steps without vanishing, so the model learns long-range patterns.
Meet the GRU
The GRU is a lighter cousin: it merges gates and drops the separate cell state, giving similar power with fewer parameters.
GRU's Two Gates
A GRU uses just two gates, a reset gate and an update gate, to balance old memory against new input each step.
Drop-In in PyTorch
Both are one-liners in PyTorch. Swap nn.LSTM or nn.GRU for nn.RNN and keep almost the same training code.
lstm = nn.LSTM(input_size=10, hidden_size=20)Which to Choose?
GRUs are faster and often match LSTMs; LSTMs can edge ahead on the hardest long sequences. Try both and let your validation score decide.
Quick Check
What is the job of the forget gate in an LSTM?
Recap
LSTMs and GRUs use gates to control memory, letting gradients survive long sequences and capturing far-apart dependencies. ✅
Frequently asked questions
Is the “LSTM & GRU Gates” lesson free?
Yes — the full text of “LSTM & GRU Gates” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “LSTM & GRU Gates”?
Remember long-range dependencies. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “LSTM & GRU Gates” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.