How an RNN Reads a Sequence
Hidden state carries the past.
How an RNN Reads a Sequence is a free NLP Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
One Word at a Time
A recurrent neural network reads text like you do, taking one token, then the next, never the whole sentence in one gulp.
The Hidden State
At each step the RNN keeps a small memory called the hidden state, a vector summarizing everything it has read so far.
A Loop Over Tokens
The same cell runs again and again, once per word. That repeated, looping structure is exactly why we call it recurrent.
Memory Carries Forward
Each new word updates the hidden state, so context from earlier in the sentence quietly flows forward to later steps.
The Update Rule
Conceptually the new state mixes the previous state with the current input through learned weights and an activation.
h = tanh(W_h @ h_prev + W_x @ x_t + b)Sharing Weights
The RNN reuses the same weights at every position. That sharing lets it handle sentences of any length with one compact cell.
Inputs Are Vectors
Each word first becomes a numeric embedding. The RNN never sees letters, only these dense vectors flowing in one by one.
A Tiny Walkthrough
Reading I love this, the state shifts after I, again after love, and the final state reflects the full short phrase.
The Final State
For classification, the hidden state after the last word is often used as a summary of the entire sequence.
Outputs Per Step
You can also read an output at every step, which is handy for tagging each word, like part-of-speech labels.
Why This Helps NLP
Because state carries the past, an RNN can finally notice that not flips the meaning of the word that follows it. 🔁
Quick Check
What does an RNN carry from one token to the next?
Recap
An RNN loops over tokens, updating a hidden state that remembers the past, so order and context finally influence its output. 🎯
Frequently asked questions
Is the “How an RNN Reads a Sequence” lesson free?
Yes — the full text of “How an RNN Reads a Sequence” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “How an RNN Reads a Sequence”?
Hidden state carries the past. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “How an RNN Reads a Sequence” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Order Matters in Language
- How an RNN Reads a Sequence
- Building an RNN Text Model
- The Vanishing Gradient Problem