Machine Learning Academy · Leçon

RNN classiques : état caché et déroulement des séquences

Vous implémenterez manuellement une cellule RNN à une étape, la déroulerez sur une courte séquence et visualiserez comment l’état caché accumule le contexte.

Leçon 1 sur 413 étapes

RNN classiques : état caché et déroulement des séquences est une leçon Machine Learning Academy gratuite sur CoddyKit. Ceci est la leçon 1 sur 4. Tu peux lire la leçon complète ci-dessous gratuitement — puis la pratiquer en direct dans le navigateur avec un éditeur de code intégré et un tuteur IA 24/7. Elle fait partie du parcours d'apprentissage Machine Learning Academy, et ta progression se synchronise sur le web et l'application CoddyKit. Le cours Machine Learning Academy comprend 4 leçons au total.

Certaines parties de cette leçon n'ont pas encore été traduites et s'affichent en anglais.

Why We Need Recurrent Networks

Standard feedforward networks treat each input independently — they have no memory of previous inputs. But many real-world problems involve sequential data where context from the past matters: predicting the next word in a sentence, forecasting tomorrow's stock price from historical data, or classifying a gesture from a video frame sequence. Recurrent Neural Networks (RNNs) solve this by maintaining a hidden state that carries information across timesteps.

# Examples of sequential data:
sequences = {
    'NLP': 'The cat sat on the ___  (predict next word)',
    'Time Series': '[1.2, 1.5, 1.3, 1.8, ???]',
    'Speech': '[audio_t0, audio_t1, ..., audio_tN]',
    'Video': '[frame_1, frame_2, ..., frame_T]',
    'DNA': 'ATCGATCG... (biological sequence)',
}
for name, example in sequences.items():
    print(f'{name}: {example}')

The Vanilla RNN Cell

A vanilla RNN cell takes two inputs: the current input x_t and the previous hidden state h_{t-1}. It produces the next hidden state h_t using the formula: h_t = tanh(W_hh * h_{t-1} + W_xh * x_t + b). The same weight matrices W_hh and W_xh are used at every timestep — this is weight sharing across time, analogous to how CNNs share weights across space. The hidden state carries the network's memory of all past inputs.

import torch

def rnn_cell(x_t, h_prev, W_xh, W_hh, b):
    '''One step of a vanilla RNN cell'''
    # x_t: (batch, input_size)
    # h_prev: (batch, hidden_size)
    h_t = torch.tanh(
        x_t @ W_xh.T +    # input contribution
        h_prev @ W_hh.T +  # hidden-to-hidden contribution
        b                  # bias
    )
    return h_t

# Example: input_size=4, hidden_size=8
batch = 3
x_t   = torch.randn(batch, 4)
h_prev = torch.zeros(batch, 8)
W_xh  = torch.randn(8, 4) * 0.01
W_hh  = torch.randn(8, 8) * 0.01
b     = torch.zeros(8)
h_t = rnn_cell(x_t, h_prev, W_xh, W_hh, b)
print(h_t.shape)   # (3, 8)

Unrolling the RNN Through Time

To process a sequence of length T, the RNN cell is applied T times in a loop — this is called unrolling (or unfolding) through time. The hidden state from step t-1 is passed to step t, connecting all timesteps. During backpropagation, gradients must flow back through every timestep — this is called Backpropagation Through Time (BPTT). The depth of this unrolled graph equals the sequence length, creating challenges for long sequences.

import torch
import torch.nn as nn

# Unroll RNN manually over a sequence
batch_size, seq_len, input_size, hidden_size = 4, 10, 8, 16

rnn_cell = nn.RNNCell(input_size, hidden_size)
sequence = torch.randn(seq_len, batch_size, input_size)

h = torch.zeros(batch_size, hidden_size)  # initial hidden state
hidden_states = []

for t in range(seq_len):
    x_t = sequence[t]   # (batch, input_size)
    h = rnn_cell(x_t, h)  # apply RNN cell
    hidden_states.append(h)

print('Final hidden state:', h.shape)     # (4, 16)
print('All hidden states:', len(hidden_states), 'steps')

Using nn.RNN: The Module Version

nn.RNN handles the unrolling automatically. It takes inputs of shape (seq_len, batch, input_size) (or (batch, seq_len, input_size) with batch_first=True) and returns all hidden states and the final hidden state. Key parameters: num_layers stacks multiple RNN layers; bidirectional=True processes the sequence in both directions; dropout applies dropout between layers in multi-layer RNNs.

import torch
import torch.nn as nn

rnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=2,
    batch_first=True,    # input: (batch, seq, features)
    dropout=0.2          # between layers
)

X = torch.randn(8, 20, 16)   # batch=8, seq=20, features=16
output, h_n = rnn(X)

print('Output shape:', output.shape)   # (8, 20, 32) all steps
print('h_n shape:', h_n.shape)         # (2, 8, 32) last hidden

Hidden State Initialisation

The initial hidden state h_0 is passed as the second argument to nn.RNN. If not provided, it defaults to zeros. For sequence classification, the final hidden state h_n summarises the entire sequence. For sequence-to-sequence tasks, all intermediate hidden states in output are used. Initialising h_0 with zeros is standard; some applications learn the initial state as a parameter for improved performance on short sequences.

import torch
import torch.nn as nn

rnn = nn.RNN(8, 16, batch_first=True)
X = torch.randn(4, 10, 8)   # batch=4, seq=10, input=8

# Default: h_0 = zeros
output, h_n = rnn(X)
print('h_n with zero init:', h_n.shape)  # (1, 4, 16)

# Custom initial hidden state
h_0 = torch.randn(1, 4, 16)  # (num_layers, batch, hidden)
output, h_n = rnn(X, h_0)
print('h_n with custom init:', h_n.shape)  # (1, 4, 16)

Sequence Classification with Final Hidden State

A common RNN application is sequence classification: given a sequence, predict a single class label. The standard approach is to use only the final hidden state h_n as the input to a linear classifier, since it has seen all the sequence so far. For a bidirectional RNN, concatenate the final forward and backward hidden states to capture both past and future context.

import torch
import torch.nn as nn

class SequenceClassifier(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        _, h_n = self.rnn(x)
        # h_n shape: (1, batch, hidden) -> squeeze to (batch, hidden)
        h_n = h_n.squeeze(0)
        return self.fc(h_n)

model = SequenceClassifier(16, 32, 5)
X = torch.randn(8, 20, 16)
logits = model(X)
print(logits.shape)   # (8, 5) -- 8 samples, 5 classes

Many-to-Many: Sequence Labelling

In sequence labelling tasks (POS tagging, named entity recognition, time-series anomaly detection), you need a prediction at every timestep, not just the last one. Use the full output tensor from nn.RNN (shape: batch x seq_len x hidden_size) and apply a linear layer to each timestep independently. The linear layer weights are shared across timesteps — one more example of weight sharing in sequence models.

import torch
import torch.nn as nn

class SequenceLabeler(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        output, _ = self.rnn(x)  # (batch, seq, hidden)
        return self.fc(output)   # (batch, seq, num_classes)

model = SequenceLabeler(8, 16, 3)
X = torch.randn(4, 10, 8)       # 4 seqs of 10 timesteps
logits = model(X)
print(logits.shape)              # (4, 10, 3) per-step labels

Stacked and Bidirectional RNNs

Stacked RNNs (num_layers > 1) feed the output of one RNN layer as input to the next, learning progressively more abstract temporal representations. Bidirectional RNNs process the sequence both forward (left to right) and backward (right to left) simultaneously, then concatenate the hidden states. Bidirectional processing allows each output to incorporate context from both past and future — useful for sentence understanding where future words clarify the meaning of past words.

import torch
import torch.nn as nn

# Stacked bidirectional RNN
brnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=3,          # 3 stacked layers
    batch_first=True,
    bidirectional=True     # forward + backward
)

X = torch.randn(4, 10, 16)
output, h_n = brnn(X)

# Output: (batch, seq, hidden*2) because bidirectional
print('Output shape:', output.shape)  # (4, 10, 64)
# h_n: (num_layers*2, batch, hidden) -- 2 dirs x 3 layers
print('h_n shape:', h_n.shape)        # (6, 4, 32)

Backpropagation Through Time (BPTT)

Backpropagation Through Time unrolls the RNN and applies standard backpropagation through the unrolled graph. For a sequence of length T, gradients are computed at every timestep and propagated backward. The gradient of the loss with respect to parameters involves products of the same weight matrix W_hh taken T times. When the spectral radius of W_hh is less than 1, these products vanish; when greater than 1, they explode — this is the fundamental challenge of training RNNs on long sequences.

import torch
import torch.nn as nn

# Demonstrate gradient flow through different sequence lengths
rnn = nn.RNNCell(4, 8)
h = torch.zeros(1, 8, requires_grad=True)

# Short sequence: gradients flow back relatively well
for t in range(5):
    x = torch.randn(1, 4)
    h = rnn(x, h)
loss = h.sum()
loss.backward()
print('h.grad (seq=5):', h.grad.norm().item())

# For long sequences (T=100+), vanilla RNN gradients
# typically vanish (near zero) or explode (very large)
# This is why LSTM/GRU were invented

Handling Variable-Length Sequences

In practice, sequences in a batch have different lengths (sentences of different word counts). PyTorch handles this with packed sequences: torch.nn.utils.rnn.pack_padded_sequence removes the padding from a batch and packs sequences efficiently. After the RNN, pad_packed_sequence restores the padded format. Without packing, the RNN processes padding tokens unnecessarily and may pollute the hidden state with meaningless padding information.

import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

rnn = nn.RNN(4, 8, batch_first=True)

# Padded sequences: length 5, 3, 2
X = torch.zeros(3, 5, 4)  # (batch=3, max_seq=5, features=4)
X[0, :, :] = torch.randn(5, 4)  # full seq
X[1, :3, :] = torch.randn(3, 4) # length 3
X[2, :2, :] = torch.randn(2, 4) # length 2
lengths = torch.tensor([5, 3, 2])

packed = pack_padded_sequence(X, lengths, batch_first=True)
out_packed, h_n = rnn(packed)
out, _ = pad_packed_sequence(out_packed, batch_first=True)
print(out.shape)  # (3, 5, 8) -- back to padded form

When to Use Vanilla RNNs

Vanilla RNNs are rarely used in practice because they suffer severely from vanishing gradients on sequences longer than 10-20 timesteps. They are primarily useful for educational purposes and very short sequences. For any real application with sequences longer than 20 timesteps, use LSTM or GRU, which have gating mechanisms specifically designed to preserve information over long distances. For sequences where order is less critical, Transformer architectures often outperform both.

# When to use each sequence model:
use_cases = {
    'Vanilla RNN (nn.RNN)':  'Short sequences (<20 steps), learning/demos',
    'LSTM':                  'Long sequences, NLP, time-series (general)',
    'GRU':                   'Similar to LSTM but faster, fewer params',
    'Transformer':           'Parallelisable, long documents, state-of-art NLP',
    'Temporal Conv (TCN)':   'Long sequences, strong parallelism, audio',
}
for model, use in use_cases.items():
    print(f'{model}: {use}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Réponds-y dans la leçon interactive

Lesson Recap

In this lesson you learned: vanilla RNNs maintain a hidden state that carries memory across timesteps using the formula h_t = tanh(W_xh * x_t + W_hh * h_{t-1} + b), unrolling applies the same cell at every timestep with shared weights, and BPTT propagates gradients back through the unrolled graph, causing vanishing/exploding gradient problems on long sequences. Next up we examine the vanishing gradient problem in depth and understand why LSTM was designed to solve it.

Gratuit pour commencer

Apprends Python avec un tuteur IA — gratuit

Écris et exécute du vrai code dans ton navigateur, obtiens de l'aide instantanée d'un tuteur IA disponible 24h/24, et reprends là où tu t'es arrêté sur le web ou dans l'app.

Cours: 30
Leçons: 120

Commencer gratuitement Voir le cours complet

Questions Fréquemment Posées

La leçon « RNN classiques : état caché et déroulement des séquences » est-elle gratuite ?

Oui — le texte complet de « RNN classiques : état caché et déroulement des séquences » est gratuit à lire ici sur le web. Pour la pratiquer de manière interactive (un éditeur de code intégré et un tuteur IA 24/7) et déverrouiller le reste du cours Machine Learning Academy, passe à CoddyKit PRO. Le cours Machine Learning Academy comprend 4 leçons au total.

Qu'est-ce que j'apprendrai dans « RNN classiques : état caché et déroulement des séquences » ?

Vous implémenterez manuellement une cellule RNN à une étape, la déroulerez sur une courte séquence et visualiserez comment l’état caché accumule le contexte. Tu pratiques Machine Learning Academy avec du code pratique que tu exécutes directement dans le navigateur, et un tuteur IA 24/7 répond à tes questions au fur et à mesure que tu avances dans la leçon.

Dois-je avoir de l'expérience pour commencer Machine Learning Academy ?

Aucune expérience préalable n'est requise. Machine Learning Academy sur CoddyKit est structuré pour les débutants jusqu'aux apprenants avancés, donc tu peux commencer ici ou depuis le début et avancer à ton rythme. Ceci est la leçon 1 sur 4.

Combien de temps prend la leçon « RNN classiques : état caché et déroulement des séquences » ?

La plupart des leçons CoddyKit prennent environ 5–10 minutes. Chacune est courte et interactive, tu progresses régulièrement et tu repiques exactement où tu t'es arrêté sur le web et l'app.

Peux-tu écrire et exécuter du code dans cette leçon Machine Learning Academy ?

Oui. Chaque leçon Machine Learning Academy inclut un éditeur de code intégré, tu écris et exécutes du vrai code directement dans ton navigateur et tu reçois des retours IA instantanés — aucune configuration locale requise.

Toutes les leçons de ce cours

RNN classiques : état caché et déroulement des séquences
Le problème de disparition des gradients sur de longues séquences
Cellule LSTM : portes d’entrée, d’oubli et de sortie
De la séquence vers un élément : analyse des sentiments avec un LSTM

← Retour à Machine Learning Academy