0Pricing
Machine Learning Academy · Aula

RNNs básicas: estado oculto e desenrolamento de sequências

Os alunos implementarão manualmente uma célula RNN de uma etapa, a desenrolarão em uma sequência curta e visualizarão como o estado oculto acumula contexto.

RNNs básicas: estado oculto e desenrolamento de sequências é uma aula grátis de Machine Learning Academy no CoddyKit. Esta é a aula 1 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de Machine Learning Academy, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de Machine Learning Academy inclui 4 aulas no total.

Partes desta aula ainda não foram traduzidas e aparecem em inglês.

Why We Need Recurrent Networks

Standard feedforward networks treat each input independently — they have no memory of previous inputs. But many real-world problems involve sequential data where context from the past matters: predicting the next word in a sentence, forecasting tomorrow's stock price from historical data, or classifying a gesture from a video frame sequence. Recurrent Neural Networks (RNNs) solve this by maintaining a hidden state that carries information across timesteps.

# Examples of sequential data:
sequences = {
    'NLP': 'The cat sat on the ___  (predict next word)',
    'Time Series': '[1.2, 1.5, 1.3, 1.8, ???]',
    'Speech': '[audio_t0, audio_t1, ..., audio_tN]',
    'Video': '[frame_1, frame_2, ..., frame_T]',
    'DNA': 'ATCGATCG... (biological sequence)',
}
for name, example in sequences.items():
    print(f'{name}: {example}')

The Vanilla RNN Cell

A vanilla RNN cell takes two inputs: the current input x_t and the previous hidden state h_{t-1}. It produces the next hidden state h_t using the formula: h_t = tanh(W_hh * h_{t-1} + W_xh * x_t + b). The same weight matrices W_hh and W_xh are used at every timestep — this is weight sharing across time, analogous to how CNNs share weights across space. The hidden state carries the network's memory of all past inputs.

import torch

def rnn_cell(x_t, h_prev, W_xh, W_hh, b):
    '''One step of a vanilla RNN cell'''
    # x_t: (batch, input_size)
    # h_prev: (batch, hidden_size)
    h_t = torch.tanh(
        x_t @ W_xh.T +    # input contribution
        h_prev @ W_hh.T +  # hidden-to-hidden contribution
        b                  # bias
    )
    return h_t

# Example: input_size=4, hidden_size=8
batch = 3
x_t   = torch.randn(batch, 4)
h_prev = torch.zeros(batch, 8)
W_xh  = torch.randn(8, 4) * 0.01
W_hh  = torch.randn(8, 8) * 0.01
b     = torch.zeros(8)
h_t = rnn_cell(x_t, h_prev, W_xh, W_hh, b)
print(h_t.shape)   # (3, 8)

Unrolling the RNN Through Time

To process a sequence of length T, the RNN cell is applied T times in a loop — this is called unrolling (or unfolding) through time. The hidden state from step t-1 is passed to step t, connecting all timesteps. During backpropagation, gradients must flow back through every timestep — this is called Backpropagation Through Time (BPTT). The depth of this unrolled graph equals the sequence length, creating challenges for long sequences.

import torch
import torch.nn as nn

# Unroll RNN manually over a sequence
batch_size, seq_len, input_size, hidden_size = 4, 10, 8, 16

rnn_cell = nn.RNNCell(input_size, hidden_size)
sequence = torch.randn(seq_len, batch_size, input_size)

h = torch.zeros(batch_size, hidden_size)  # initial hidden state
hidden_states = []

for t in range(seq_len):
    x_t = sequence[t]   # (batch, input_size)
    h = rnn_cell(x_t, h)  # apply RNN cell
    hidden_states.append(h)

print('Final hidden state:', h.shape)     # (4, 16)
print('All hidden states:', len(hidden_states), 'steps')

Using nn.RNN: The Module Version

nn.RNN handles the unrolling automatically. It takes inputs of shape (seq_len, batch, input_size) (or (batch, seq_len, input_size) with batch_first=True) and returns all hidden states and the final hidden state. Key parameters: num_layers stacks multiple RNN layers; bidirectional=True processes the sequence in both directions; dropout applies dropout between layers in multi-layer RNNs.

import torch
import torch.nn as nn

rnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=2,
    batch_first=True,    # input: (batch, seq, features)
    dropout=0.2          # between layers
)

X = torch.randn(8, 20, 16)   # batch=8, seq=20, features=16
output, h_n = rnn(X)

print('Output shape:', output.shape)   # (8, 20, 32) all steps
print('h_n shape:', h_n.shape)         # (2, 8, 32) last hidden

Hidden State Initialisation

The initial hidden state h_0 is passed as the second argument to nn.RNN. If not provided, it defaults to zeros. For sequence classification, the final hidden state h_n summarises the entire sequence. For sequence-to-sequence tasks, all intermediate hidden states in output are used. Initialising h_0 with zeros is standard; some applications learn the initial state as a parameter for improved performance on short sequences.

import torch
import torch.nn as nn

rnn = nn.RNN(8, 16, batch_first=True)
X = torch.randn(4, 10, 8)   # batch=4, seq=10, input=8

# Default: h_0 = zeros
output, h_n = rnn(X)
print('h_n with zero init:', h_n.shape)  # (1, 4, 16)

# Custom initial hidden state
h_0 = torch.randn(1, 4, 16)  # (num_layers, batch, hidden)
output, h_n = rnn(X, h_0)
print('h_n with custom init:', h_n.shape)  # (1, 4, 16)

Sequence Classification with Final Hidden State

A common RNN application is sequence classification: given a sequence, predict a single class label. The standard approach is to use only the final hidden state h_n as the input to a linear classifier, since it has seen all the sequence so far. For a bidirectional RNN, concatenate the final forward and backward hidden states to capture both past and future context.

import torch
import torch.nn as nn

class SequenceClassifier(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        _, h_n = self.rnn(x)
        # h_n shape: (1, batch, hidden) -> squeeze to (batch, hidden)
        h_n = h_n.squeeze(0)
        return self.fc(h_n)

model = SequenceClassifier(16, 32, 5)
X = torch.randn(8, 20, 16)
logits = model(X)
print(logits.shape)   # (8, 5) -- 8 samples, 5 classes

Many-to-Many: Sequence Labelling

In sequence labelling tasks (POS tagging, named entity recognition, time-series anomaly detection), you need a prediction at every timestep, not just the last one. Use the full output tensor from nn.RNN (shape: batch x seq_len x hidden_size) and apply a linear layer to each timestep independently. The linear layer weights are shared across timesteps — one more example of weight sharing in sequence models.

import torch
import torch.nn as nn

class SequenceLabeler(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        output, _ = self.rnn(x)  # (batch, seq, hidden)
        return self.fc(output)   # (batch, seq, num_classes)

model = SequenceLabeler(8, 16, 3)
X = torch.randn(4, 10, 8)       # 4 seqs of 10 timesteps
logits = model(X)
print(logits.shape)              # (4, 10, 3) per-step labels

Stacked and Bidirectional RNNs

Stacked RNNs (num_layers > 1) feed the output of one RNN layer as input to the next, learning progressively more abstract temporal representations. Bidirectional RNNs process the sequence both forward (left to right) and backward (right to left) simultaneously, then concatenate the hidden states. Bidirectional processing allows each output to incorporate context from both past and future — useful for sentence understanding where future words clarify the meaning of past words.

import torch
import torch.nn as nn

# Stacked bidirectional RNN
brnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=3,          # 3 stacked layers
    batch_first=True,
    bidirectional=True     # forward + backward
)

X = torch.randn(4, 10, 16)
output, h_n = brnn(X)

# Output: (batch, seq, hidden*2) because bidirectional
print('Output shape:', output.shape)  # (4, 10, 64)
# h_n: (num_layers*2, batch, hidden) -- 2 dirs x 3 layers
print('h_n shape:', h_n.shape)        # (6, 4, 32)

Backpropagation Through Time (BPTT)

Backpropagation Through Time unrolls the RNN and applies standard backpropagation through the unrolled graph. For a sequence of length T, gradients are computed at every timestep and propagated backward. The gradient of the loss with respect to parameters involves products of the same weight matrix W_hh taken T times. When the spectral radius of W_hh is less than 1, these products vanish; when greater than 1, they explode — this is the fundamental challenge of training RNNs on long sequences.

import torch
import torch.nn as nn

# Demonstrate gradient flow through different sequence lengths
rnn = nn.RNNCell(4, 8)
h = torch.zeros(1, 8, requires_grad=True)

# Short sequence: gradients flow back relatively well
for t in range(5):
    x = torch.randn(1, 4)
    h = rnn(x, h)
loss = h.sum()
loss.backward()
print('h.grad (seq=5):', h.grad.norm().item())

# For long sequences (T=100+), vanilla RNN gradients
# typically vanish (near zero) or explode (very large)
# This is why LSTM/GRU were invented

Handling Variable-Length Sequences

In practice, sequences in a batch have different lengths (sentences of different word counts). PyTorch handles this with packed sequences: torch.nn.utils.rnn.pack_padded_sequence removes the padding from a batch and packs sequences efficiently. After the RNN, pad_packed_sequence restores the padded format. Without packing, the RNN processes padding tokens unnecessarily and may pollute the hidden state with meaningless padding information.

import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

rnn = nn.RNN(4, 8, batch_first=True)

# Padded sequences: length 5, 3, 2
X = torch.zeros(3, 5, 4)  # (batch=3, max_seq=5, features=4)
X[0, :, :] = torch.randn(5, 4)  # full seq
X[1, :3, :] = torch.randn(3, 4) # length 3
X[2, :2, :] = torch.randn(2, 4) # length 2
lengths = torch.tensor([5, 3, 2])

packed = pack_padded_sequence(X, lengths, batch_first=True)
out_packed, h_n = rnn(packed)
out, _ = pad_packed_sequence(out_packed, batch_first=True)
print(out.shape)  # (3, 5, 8) -- back to padded form

When to Use Vanilla RNNs

Vanilla RNNs are rarely used in practice because they suffer severely from vanishing gradients on sequences longer than 10-20 timesteps. They are primarily useful for educational purposes and very short sequences. For any real application with sequences longer than 20 timesteps, use LSTM or GRU, which have gating mechanisms specifically designed to preserve information over long distances. For sequences where order is less critical, Transformer architectures often outperform both.

# When to use each sequence model:
use_cases = {
    'Vanilla RNN (nn.RNN)':  'Short sequences (<20 steps), learning/demos',
    'LSTM':                  'Long sequences, NLP, time-series (general)',
    'GRU':                   'Similar to LSTM but faster, fewer params',
    'Transformer':           'Parallelisable, long documents, state-of-art NLP',
    'Temporal Conv (TCN)':   'Long sequences, strong parallelism, audio',
}
for model, use in use_cases.items():
    print(f'{model}: {use}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: vanilla RNNs maintain a hidden state that carries memory across timesteps using the formula h_t = tanh(W_xh * x_t + W_hh * h_{t-1} + b), unrolling applies the same cell at every timestep with shared weights, and BPTT propagates gradients back through the unrolled graph, causing vanishing/exploding gradient problems on long sequences. Next up we examine the vanishing gradient problem in depth and understand why LSTM was designed to solve it.

Perguntas Frequentes

A aula “RNNs básicas: estado oculto e desenrolamento de sequências” é grátis?

Sim — o texto completo de “RNNs básicas: estado oculto e desenrolamento de sequências” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de Machine Learning Academy, atualize para CoddyKit PRO. O curso de Machine Learning Academy inclui 4 aulas no total.

O que vou aprender em “RNNs básicas: estado oculto e desenrolamento de sequências”?

Os alunos implementarão manualmente uma célula RNN de uma etapa, a desenrolarão em uma sequência curta e visualizarão como o estado oculto acumula contexto. Você pratica Machine Learning Academy com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.

Preciso ter experiência prévia para começar Machine Learning Academy?

Nenhuma experiência prévia é necessária. Machine Learning Academy no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 1 de 4.

Quanto tempo leva a aula “RNNs básicas: estado oculto e desenrolamento de sequências”?

A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.

Posso escrever e executar código nesta aula de Machine Learning Academy?

Sim. Cada aula de Machine Learning Academy inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.

Todas as aulas deste curso

  1. RNNs básicas: estado oculto e desenrolamento de sequências
  2. O problema do gradiente que desaparece em etapas temporais profundas
  3. Célula LSTM: portas de entrada, esquecimento e saída
  4. De sequência para uma saída: análise de sentimentos com uma LSTM
← Voltar para Machine Learning Academy