0Pricing
Machine Learning Academy · Lección

RNN básicas: estado oculto y desenrollado de secuencias

Implementará manualmente una celda RNN de un paso, la desenrollará a lo largo de una secuencia corta y visualizará cómo el estado oculto acumula contexto.

RNN básicas: estado oculto y desenrollado de secuencias es una lección gratuita de Machine Learning Academy en CoddyKit. Esta es la lección 1 de 4. Puedes leer la lección completa abajo gratuitamente — luego la practicas en el navegador con un editor de código integrado y un tutor de IA 24/7. Forma parte de la ruta de aprendizaje de Machine Learning Academy, y tu progreso se sincroniza en la web y la app de CoddyKit. El curso de Machine Learning Academy incluye 4 lecciones en total.

Partes de esta lección aún no han sido traducidas y se muestran en inglés.

Why We Need Recurrent Networks

Standard feedforward networks treat each input independently — they have no memory of previous inputs. But many real-world problems involve sequential data where context from the past matters: predicting the next word in a sentence, forecasting tomorrow's stock price from historical data, or classifying a gesture from a video frame sequence. Recurrent Neural Networks (RNNs) solve this by maintaining a hidden state that carries information across timesteps.

# Examples of sequential data:
sequences = {
    'NLP': 'The cat sat on the ___  (predict next word)',
    'Time Series': '[1.2, 1.5, 1.3, 1.8, ???]',
    'Speech': '[audio_t0, audio_t1, ..., audio_tN]',
    'Video': '[frame_1, frame_2, ..., frame_T]',
    'DNA': 'ATCGATCG... (biological sequence)',
}
for name, example in sequences.items():
    print(f'{name}: {example}')

The Vanilla RNN Cell

A vanilla RNN cell takes two inputs: the current input x_t and the previous hidden state h_{t-1}. It produces the next hidden state h_t using the formula: h_t = tanh(W_hh * h_{t-1} + W_xh * x_t + b). The same weight matrices W_hh and W_xh are used at every timestep — this is weight sharing across time, analogous to how CNNs share weights across space. The hidden state carries the network's memory of all past inputs.

import torch

def rnn_cell(x_t, h_prev, W_xh, W_hh, b):
    '''One step of a vanilla RNN cell'''
    # x_t: (batch, input_size)
    # h_prev: (batch, hidden_size)
    h_t = torch.tanh(
        x_t @ W_xh.T +    # input contribution
        h_prev @ W_hh.T +  # hidden-to-hidden contribution
        b                  # bias
    )
    return h_t

# Example: input_size=4, hidden_size=8
batch = 3
x_t   = torch.randn(batch, 4)
h_prev = torch.zeros(batch, 8)
W_xh  = torch.randn(8, 4) * 0.01
W_hh  = torch.randn(8, 8) * 0.01
b     = torch.zeros(8)
h_t = rnn_cell(x_t, h_prev, W_xh, W_hh, b)
print(h_t.shape)   # (3, 8)

Unrolling the RNN Through Time

To process a sequence of length T, the RNN cell is applied T times in a loop — this is called unrolling (or unfolding) through time. The hidden state from step t-1 is passed to step t, connecting all timesteps. During backpropagation, gradients must flow back through every timestep — this is called Backpropagation Through Time (BPTT). The depth of this unrolled graph equals the sequence length, creating challenges for long sequences.

import torch
import torch.nn as nn

# Unroll RNN manually over a sequence
batch_size, seq_len, input_size, hidden_size = 4, 10, 8, 16

rnn_cell = nn.RNNCell(input_size, hidden_size)
sequence = torch.randn(seq_len, batch_size, input_size)

h = torch.zeros(batch_size, hidden_size)  # initial hidden state
hidden_states = []

for t in range(seq_len):
    x_t = sequence[t]   # (batch, input_size)
    h = rnn_cell(x_t, h)  # apply RNN cell
    hidden_states.append(h)

print('Final hidden state:', h.shape)     # (4, 16)
print('All hidden states:', len(hidden_states), 'steps')

Using nn.RNN: The Module Version

nn.RNN handles the unrolling automatically. It takes inputs of shape (seq_len, batch, input_size) (or (batch, seq_len, input_size) with batch_first=True) and returns all hidden states and the final hidden state. Key parameters: num_layers stacks multiple RNN layers; bidirectional=True processes the sequence in both directions; dropout applies dropout between layers in multi-layer RNNs.

import torch
import torch.nn as nn

rnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=2,
    batch_first=True,    # input: (batch, seq, features)
    dropout=0.2          # between layers
)

X = torch.randn(8, 20, 16)   # batch=8, seq=20, features=16
output, h_n = rnn(X)

print('Output shape:', output.shape)   # (8, 20, 32) all steps
print('h_n shape:', h_n.shape)         # (2, 8, 32) last hidden

Hidden State Initialisation

The initial hidden state h_0 is passed as the second argument to nn.RNN. If not provided, it defaults to zeros. For sequence classification, the final hidden state h_n summarises the entire sequence. For sequence-to-sequence tasks, all intermediate hidden states in output are used. Initialising h_0 with zeros is standard; some applications learn the initial state as a parameter for improved performance on short sequences.

import torch
import torch.nn as nn

rnn = nn.RNN(8, 16, batch_first=True)
X = torch.randn(4, 10, 8)   # batch=4, seq=10, input=8

# Default: h_0 = zeros
output, h_n = rnn(X)
print('h_n with zero init:', h_n.shape)  # (1, 4, 16)

# Custom initial hidden state
h_0 = torch.randn(1, 4, 16)  # (num_layers, batch, hidden)
output, h_n = rnn(X, h_0)
print('h_n with custom init:', h_n.shape)  # (1, 4, 16)

Sequence Classification with Final Hidden State

A common RNN application is sequence classification: given a sequence, predict a single class label. The standard approach is to use only the final hidden state h_n as the input to a linear classifier, since it has seen all the sequence so far. For a bidirectional RNN, concatenate the final forward and backward hidden states to capture both past and future context.

import torch
import torch.nn as nn

class SequenceClassifier(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        _, h_n = self.rnn(x)
        # h_n shape: (1, batch, hidden) -> squeeze to (batch, hidden)
        h_n = h_n.squeeze(0)
        return self.fc(h_n)

model = SequenceClassifier(16, 32, 5)
X = torch.randn(8, 20, 16)
logits = model(X)
print(logits.shape)   # (8, 5) -- 8 samples, 5 classes

Many-to-Many: Sequence Labelling

In sequence labelling tasks (POS tagging, named entity recognition, time-series anomaly detection), you need a prediction at every timestep, not just the last one. Use the full output tensor from nn.RNN (shape: batch x seq_len x hidden_size) and apply a linear layer to each timestep independently. The linear layer weights are shared across timesteps — one more example of weight sharing in sequence models.

import torch
import torch.nn as nn

class SequenceLabeler(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        self.rnn = nn.RNN(input_size, hidden_size,
                          batch_first=True)
        self.fc  = nn.Linear(hidden_size, num_classes)

    def forward(self, x):
        output, _ = self.rnn(x)  # (batch, seq, hidden)
        return self.fc(output)   # (batch, seq, num_classes)

model = SequenceLabeler(8, 16, 3)
X = torch.randn(4, 10, 8)       # 4 seqs of 10 timesteps
logits = model(X)
print(logits.shape)              # (4, 10, 3) per-step labels

Stacked and Bidirectional RNNs

Stacked RNNs (num_layers > 1) feed the output of one RNN layer as input to the next, learning progressively more abstract temporal representations. Bidirectional RNNs process the sequence both forward (left to right) and backward (right to left) simultaneously, then concatenate the hidden states. Bidirectional processing allows each output to incorporate context from both past and future — useful for sentence understanding where future words clarify the meaning of past words.

import torch
import torch.nn as nn

# Stacked bidirectional RNN
brnn = nn.RNN(
    input_size=16,
    hidden_size=32,
    num_layers=3,          # 3 stacked layers
    batch_first=True,
    bidirectional=True     # forward + backward
)

X = torch.randn(4, 10, 16)
output, h_n = brnn(X)

# Output: (batch, seq, hidden*2) because bidirectional
print('Output shape:', output.shape)  # (4, 10, 64)
# h_n: (num_layers*2, batch, hidden) -- 2 dirs x 3 layers
print('h_n shape:', h_n.shape)        # (6, 4, 32)

Backpropagation Through Time (BPTT)

Backpropagation Through Time unrolls the RNN and applies standard backpropagation through the unrolled graph. For a sequence of length T, gradients are computed at every timestep and propagated backward. The gradient of the loss with respect to parameters involves products of the same weight matrix W_hh taken T times. When the spectral radius of W_hh is less than 1, these products vanish; when greater than 1, they explode — this is the fundamental challenge of training RNNs on long sequences.

import torch
import torch.nn as nn

# Demonstrate gradient flow through different sequence lengths
rnn = nn.RNNCell(4, 8)
h = torch.zeros(1, 8, requires_grad=True)

# Short sequence: gradients flow back relatively well
for t in range(5):
    x = torch.randn(1, 4)
    h = rnn(x, h)
loss = h.sum()
loss.backward()
print('h.grad (seq=5):', h.grad.norm().item())

# For long sequences (T=100+), vanilla RNN gradients
# typically vanish (near zero) or explode (very large)
# This is why LSTM/GRU were invented

Handling Variable-Length Sequences

In practice, sequences in a batch have different lengths (sentences of different word counts). PyTorch handles this with packed sequences: torch.nn.utils.rnn.pack_padded_sequence removes the padding from a batch and packs sequences efficiently. After the RNN, pad_packed_sequence restores the padded format. Without packing, the RNN processes padding tokens unnecessarily and may pollute the hidden state with meaningless padding information.

import torch
import torch.nn as nn
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

rnn = nn.RNN(4, 8, batch_first=True)

# Padded sequences: length 5, 3, 2
X = torch.zeros(3, 5, 4)  # (batch=3, max_seq=5, features=4)
X[0, :, :] = torch.randn(5, 4)  # full seq
X[1, :3, :] = torch.randn(3, 4) # length 3
X[2, :2, :] = torch.randn(2, 4) # length 2
lengths = torch.tensor([5, 3, 2])

packed = pack_padded_sequence(X, lengths, batch_first=True)
out_packed, h_n = rnn(packed)
out, _ = pad_packed_sequence(out_packed, batch_first=True)
print(out.shape)  # (3, 5, 8) -- back to padded form

When to Use Vanilla RNNs

Vanilla RNNs are rarely used in practice because they suffer severely from vanishing gradients on sequences longer than 10-20 timesteps. They are primarily useful for educational purposes and very short sequences. For any real application with sequences longer than 20 timesteps, use LSTM or GRU, which have gating mechanisms specifically designed to preserve information over long distances. For sequences where order is less critical, Transformer architectures often outperform both.

# When to use each sequence model:
use_cases = {
    'Vanilla RNN (nn.RNN)':  'Short sequences (<20 steps), learning/demos',
    'LSTM':                  'Long sequences, NLP, time-series (general)',
    'GRU':                   'Similar to LSTM but faster, fewer params',
    'Transformer':           'Parallelisable, long documents, state-of-art NLP',
    'Temporal Conv (TCN)':   'Long sequences, strong parallelism, audio',
}
for model, use in use_cases.items():
    print(f'{model}: {use}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: vanilla RNNs maintain a hidden state that carries memory across timesteps using the formula h_t = tanh(W_xh * x_t + W_hh * h_{t-1} + b), unrolling applies the same cell at every timestep with shared weights, and BPTT propagates gradients back through the unrolled graph, causing vanishing/exploding gradient problems on long sequences. Next up we examine the vanishing gradient problem in depth and understand why LSTM was designed to solve it.

Preguntas frecuentes

¿La lección «RNN básicas: estado oculto y desenrollado de secuencias» es gratis?

Sí — el texto completo de «RNN básicas: estado oculto y desenrollado de secuencias» es gratis para leer aquí en la web. Para practicarla de forma interactiva (editor de código integrado y tutor de IA 24/7) y desbloquear el resto del curso de Machine Learning Academy, actualiza a CoddyKit PRO. El curso de Machine Learning Academy incluye 4 lecciones en total.

¿Qué aprenderé en «RNN básicas: estado oculto y desenrollado de secuencias»?

Implementará manualmente una celda RNN de un paso, la desenrollará a lo largo de una secuencia corta y visualizará cómo el estado oculto acumula contexto. Practicas Machine Learning Academy con código real que ejecutas directamente en el navegador, y un tutor de IA 24/7 responde tus preguntas mientras trabajas en la lección.

¿Necesito experiencia previa para empezar Machine Learning Academy?

No se requiere experiencia previa. Machine Learning Academy en CoddyKit está estructurado para principiantes hasta estudiantes avanzados, así que puedes empezar aquí o desde el inicio y avanzar a tu ritmo. Esta es la lección 1 de 4.

¿Cuánto tiempo toma la lección «RNN básicas: estado oculto y desenrollado de secuencias»?

La mayoría de las lecciones de CoddyKit toman alrededor de 5–10 minutos. Cada una es compacta e interactiva, así que avanzas constantemente y retomas exactamente por donde dejaste en la web y la app.

¿Puedo escribir y ejecutar código en esta lección de Machine Learning Academy?

Sí. Cada lección de Machine Learning Academy incluye un editor de código integrado, así que escribes y ejecutas código real directamente en tu navegador y obtienes retroalimentación instantánea de IA — sin configuración local necesaria.

Todas las lecciones de este curso

  1. RNN básicas: estado oculto y desenrollado de secuencias
  2. El problema del gradiente evanescente en pasos temporales profundos
  3. Celda LSTM: puertas de entrada, olvido y salida
  4. De secuencia a uno: análisis de sentimiento con una LSTM
← Volver a Machine Learning Academy