Gates That Control Memory
How LSTMs decide what to keep.
Gates That Control Memory is a free NLP Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Plain RNNs Forget
A simple RNN struggles to remember things from far back. The LSTM was built to fix that by carefully managing what it keeps and forgets. 🧠
The Cell State
An LSTM carries a special cell state, a memory line that runs straight through every step so important signals survive long sequences.
Gates Are the Trick
The magic is in the gates: tiny learned switches that decide what to add, keep, or drop from memory at each time step.
How a Gate Works
Each gate uses a sigmoid to output values from 0 to 1, acting like a dimmer that lets some information through and blocks the rest.
gate = sigmoid(W @ x + U @ h + b) # values in [0, 1]The Forget Gate
The forget gate looks at the new input and decides how much of the old memory to erase. A 0 wipes it; a 1 keeps it fully.
The Input Gate
The input gate controls how much fresh information gets written into the cell state, so only useful new signals are stored.
The Candidate Values
A candidate vector proposes what new content could enter memory. The input gate then scales how much of it actually gets added.
candidate = tanh(W @ x + U @ h + b)Updating Memory
The new cell state blends old memory kept by the forget gate with fresh candidate values let in by the input gate.
c = forget * c_prev + input * candidateThe Output Gate
The output gate decides what part of the cell state becomes the visible hidden state passed to the next step and the layer above.
Why Gates Beat Forgetting
Because gates keep the cell state mostly additive, gradients flow back many steps without vanishing. That is how an LSTM remembers long context.
Three Gates, One Cell
So an LSTM has three gates guarding one memory line: forget, input, and output, each learned from data during training.
Quick Check
Test your grip on LSTM gates.
Recap
An LSTM protects a cell state with forget, input, and output gates. Those switches keep long-range memory alive where plain RNNs fail. ✅
Frequently asked questions
Is the “Gates That Control Memory” lesson free?
Yes — the full text of “Gates That Control Memory” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Gates That Control Memory”?
How LSTMs decide what to keep. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Gates That Control Memory” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Gates That Control Memory
- GRU: A Leaner Alternative
- Training an LSTM Classifier
- Bidirectional and Stacked Layers