خلية RNN التقليدية

An RNN cell is the tiny engine that runs at each time step. You feed it one input and the old memory, and it returns the new memory.

Every step the cell takes two things: the current input x and the previous hidden state. It mixes them into a fresh hidden state.

The vanilla cell computes a weighted sum of input and memory, then squashes it. That single line is the heart of the RNN.

The tanh activation keeps the hidden state bounded between -1 and 1, so memory values stay stable instead of blowing up over many steps.

One matrix W_x reads the new input; another W_h reads the old memory. The cell learns both to decide what to keep and what to add.

Before step one there is no memory, so the hidden state starts as a vector of zeros and gets filled in as the sequence flows through.

Picture the cell copied once per step, with memory passed along the chain. This view is called unrolling the network through time.

Need a prediction at a step? Pass that step's hidden state through a small output layer to get logits or a value.

PyTorch gives you nn.RNN so you don't hand-code the loop. You set input and hidden sizes, then feed a batched sequence.

Calling the layer returns two things: the outputs at every step and the final hidden state, handy for classifying the whole sequence.

Vanilla RNNs work, but their memory fades fast over long sequences. That weakness sets the stage for gated cells like the LSTM.

What two things does a vanilla RNN cell take as input each step?

The vanilla cell blends input and old memory with tanh to make a new hidden state, repeated step by step across the sequence. ✅