Pourquoi les séquences ont besoin de mémoire

In a sequence, order is information. The words 'dog bites man' and 'man bites dog' use the same tokens but mean opposite things. 🐶

Text, audio, stock prices, and sensor readings are all sequences: ordered lists where each step relates to the steps around it.

A standard feedforward net sees one fixed-size input at a time. It has no way to know what came before, so it forgets the past instantly.

Sentences come in all sizes. A network with fixed inputs can't gracefully handle a 3-word phrase and a 30-word one with the same weights.

We need a model that keeps a running memory of what it has seen, updating that memory at every step in the sequence.

That memory is called the hidden state: a vector the model carries forward, summarizing everything important from earlier steps.

A recurrent model reads the sequence step by step, blending the new input with its current memory to produce an updated memory.

The same set of weights is reused at every time step. This keeps the model small and lets it handle sequences of any length.

Memory lets context shape predictions. To finish 'the clouds are in the ___', the model leans on the earlier word 'clouds'.

Some clues sit right next door; others are far back in the sequence. Good memory must capture both short and long range dependencies.

Models built around a looping hidden state are called recurrent networks. They are the classic answer to learning from ordered data.

Why can't a plain feedforward network model a sequence well?

Sequences carry meaning in their order, and plain nets forget the past. The fix is a model with memory that updates step by step. ✅