The Vanilla RNN Cell
Carry a hidden state across steps.
The Vanilla RNN Cell is a free Deep Learning Academy lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What a Cell Does
An RNN cell is the tiny engine that runs at each time step. You feed it one input and the old memory, and it returns the new memory.
Two Inputs Per Step
Every step the cell takes two things: the current input x and the previous hidden state. It mixes them into a fresh hidden state.
The Core Formula
The vanilla cell computes a weighted sum of input and memory, then squashes it. That single line is the heart of the RNN.
h_t = tanh(W_x @ x_t + W_h @ h_prev + b)Why tanh?
The tanh activation keeps the hidden state bounded between -1 and 1, so memory values stay stable instead of blowing up over many steps.
Two Weight Matrices
One matrix W_x reads the new input; another W_h reads the old memory. The cell learns both to decide what to keep and what to add.
The Initial Hidden State
Before step one there is no memory, so the hidden state starts as a vector of zeros and gets filled in as the sequence flows through.
h0 = torch.zeros(hidden_size)Unrolling Through Time
Picture the cell copied once per step, with memory passed along the chain. This view is called unrolling the network through time.
Outputs From Memory
Need a prediction at a step? Pass that step's hidden state through a small output layer to get logits or a value.
y_t = W_y @ h_t + b_yUse It in PyTorch
PyTorch gives you nn.RNN so you don't hand-code the loop. You set input and hidden sizes, then feed a batched sequence.
rnn = nn.RNN(input_size=10, hidden_size=20)Outputs and Final State
Calling the layer returns two things: the outputs at every step and the final hidden state, handy for classifying the whole sequence.
out, h_n = rnn(x)The Catch
Vanilla RNNs work, but their memory fades fast over long sequences. That weakness sets the stage for gated cells like the LSTM.
Quick Check
What two things does a vanilla RNN cell take as input each step?
Recap
The vanilla cell blends input and old memory with tanh to make a new hidden state, repeated step by step across the sequence. ✅
Frequently asked questions
Is the “The Vanilla RNN Cell” lesson free?
Yes — the full text of “The Vanilla RNN Cell” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “The Vanilla RNN Cell”?
Carry a hidden state across steps. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “The Vanilla RNN Cell” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Sequences Need Memory
- The Vanilla RNN Cell
- LSTM & GRU Gates
- Pack Sequences & Handle Padding