0Pricing
Machine Learning Academy · Lektion

Trainingsschleife: Loss, Optimizer und Epochen

Lernende schreiben die PyTorch-Trainingsschleife: zero_grad, Forward-Pass, Berechnung von CrossEntropyLoss, Backward-Pass und optimizer.step, und verfolgen Loss und Genauigkeit über mehrere Epochen.

Trainingsschleife: Loss, Optimizer und Epochen ist eine kostenlose Machine Learning Academy-Lektion auf CoddyKit. Dies ist Lektion 4 von 4. Du kannst die komplette Lektion unten kostenlos lesen – dann übst du sie direkt im Browser mit einem integrierten Code-Editor und einem KI-Tutor rund um die Uhr. Sie ist Teil des Machine Learning Academy-Lernpfads, und dein Fortschritt wird über Web und CoddyKit-App synchronisiert. Der Machine Learning Academy-Kurs umfasst insgesamt 4 Lektionen.

Teile dieser Lektion wurden noch nicht übersetzt und werden auf Englisch angezeigt.

The Four Steps of Every Training Loop

The PyTorch training loop has four mandatory steps that repeat for every batch: (1) zero gradients, (2) forward pass, (3) backward pass, and (4) optimizer step. Skipping or reordering these steps produces wrong results silently — gradients accumulate, parameters update incorrectly, or memory leaks. Internalising this pattern is the most important habit for training neural networks with PyTorch.

import torch
import torch.nn as nn
import torch.optim as optim

model = nn.Linear(2, 1)
optimizer = optim.SGD(model.parameters(), lr=0.01)
criterion = nn.MSELoss()

X = torch.randn(20, 2)
y = torch.randn(20, 1)

for step in range(1):
    optimizer.zero_grad()           # (1) zero grads
    y_pred = model(X)               # (2) forward
    loss = criterion(y_pred, y)     # (2) loss
    loss.backward()                 # (3) backward
    optimizer.step()                # (4) update
    print('Loss:', loss.item())

Loss Functions: Choosing the Right Criterion

The loss function measures how wrong the model's predictions are. PyTorch's nn module provides many: nn.MSELoss for regression (mean squared error), nn.CrossEntropyLoss for multi-class classification (combines log-softmax and NLL), and nn.BCEWithLogitsLoss for binary classification (combines sigmoid and binary cross-entropy). Using the wrong loss for your task is a common beginner mistake that prevents learning.

import torch
import torch.nn as nn

# Regression
mse = nn.MSELoss()
y_pred = torch.tensor([2.5, 3.0])
y_true = torch.tensor([2.0, 3.5])
print('MSE:', mse(y_pred, y_true).item())

# Multi-class: logits (raw scores), not softmax
ce = nn.CrossEntropyLoss()
logits = torch.tensor([[2.0, 0.5, 1.0]])
labels = torch.tensor([0])
print('CE:', ce(logits, labels).item())

Optimizers: SGD, Adam, and AdamW

An optimizer uses computed gradients to update model parameters. SGD (Stochastic Gradient Descent) is the classic optimizer; adding momentum accelerates convergence. Adam adapts the learning rate per parameter using first and second moment estimates, converging faster in practice. AdamW adds proper weight decay (L2 regularisation) and is the default choice for transformer models. All optimizers are in torch.optim.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)

# Stochastic Gradient Descent with momentum
sgd = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)

# Adam: adaptive learning rate
adam = optim.Adam(model.parameters(), lr=0.001,
                  betas=(0.9, 0.999))

# AdamW: Adam + proper weight decay
adamw = optim.AdamW(model.parameters(), lr=0.001,
                    weight_decay=0.01)

Iterating Over Epochs and Batches

Training typically runs for many epochs — complete passes through the training dataset. Within each epoch you iterate over batches (subsets of the data). Using a DataLoader handles shuffling and batching automatically. Tracking average loss per epoch lets you monitor whether the model is converging. Printing every epoch (or every N batches) gives you visibility into training progress.

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn as nn
import torch.optim as optim

X = torch.randn(200, 4)
y = torch.randn(200, 1)
dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

model = nn.Linear(4, 1)
optimizer = optim.Adam(model.parameters())
criterion = nn.MSELoss()

for epoch in range(3):
    total_loss = 0
    for X_batch, y_batch in loader:
        optimizer.zero_grad()
        pred = model(X_batch)
        loss = criterion(pred, y_batch)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f'Epoch {epoch}: avg_loss={total_loss/len(loader):.4f}')

DataLoader: Batching and Shuffling Data

DataLoader wraps a Dataset and provides an iterator that yields batches. Key parameters: batch_size controls how many samples per gradient update; shuffle=True randomises order each epoch (crucial for training); num_workers enables parallel data loading. For custom datasets, subclass torch.utils.data.Dataset and implement __len__ and __getitem__.

import torch
from torch.utils.data import Dataset, DataLoader

class MyDataset(Dataset):
    def __init__(self, X, y):
        self.X = X
        self.y = y

    def __len__(self):
        return len(self.X)

    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

X = torch.randn(100, 5)
y = torch.randint(0, 3, (100,))
dataset = MyDataset(X, y)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

X_batch, y_batch = next(iter(loader))
print(X_batch.shape)   # torch.Size([16, 5])

CrossEntropyLoss: Logits and Class Indices

nn.CrossEntropyLoss expects raw logits (unnormalised scores), not softmax probabilities. It internally applies log-softmax and computes negative log-likelihood. Target labels should be integer class indices (not one-hot vectors). This is the correct loss for multi-class classification and is more numerically stable than manually applying softmax then NLLLoss. The model's final layer should NOT have a softmax activation when using this loss.

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss()

# Batch of 4 samples, 3 classes
logits = torch.tensor([
    [2.1, 0.5, 0.3],
    [0.1, 3.0, 0.2],
    [0.8, 0.9, 2.0],
    [1.5, 0.1, 0.4]
])

# True labels as class indices (not one-hot)
labels = torch.tensor([0, 1, 2, 0])

loss = criterion(logits, labels)
print('Loss:', loss.item())

Tracking Accuracy During Training

Loss decreasing confirms the model is learning, but accuracy tells you whether predictions are actually correct. For multi-class classification, convert logits to predicted class with torch.argmax along the class dimension, then compare with true labels. Tracking both training loss and training accuracy, and optionally validation accuracy, gives a complete picture of learning dynamics.

import torch

def compute_accuracy(logits, labels):
    preds = torch.argmax(logits, dim=1)
    correct = (preds == labels).sum().item()
    return correct / len(labels)

logits = torch.tensor([
    [2.0, 0.1, 0.3],
    [0.1, 0.2, 3.5],
    [1.0, 0.5, 0.2]
])
labels = torch.tensor([0, 2, 0])

print('Accuracy:', compute_accuracy(logits, labels))
# 1.0 (all three correct)

Validation Loop: Evaluating Without Updating

After each training epoch, run a validation loop to assess performance on unseen data. Wrap it with torch.no_grad() to disable gradient computation and call model.eval() to deactivate Dropout and BatchNorm stochasticity. Comparing training loss vs validation loss is the primary tool for detecting overfitting — when validation loss rises while training loss continues to fall, the model has overfit the training set.

import torch

def validate(model, val_loader, criterion):
    model.eval()
    total_loss = 0
    with torch.no_grad():
        for X_batch, y_batch in val_loader:
            logits = model(X_batch)
            loss = criterion(logits, y_batch)
            total_loss += loss.item()
    model.train()   # restore training mode
    return total_loss / len(val_loader)

# Usage inside training loop:
# val_loss = validate(model, val_loader, criterion)
# print(f'Val loss: {val_loss:.4f}')

Learning Rate Scheduling

A fixed learning rate often leads to slow convergence or oscillation near the minimum. Learning rate schedulers automatically adjust the LR during training. StepLR reduces LR by a factor every N epochs; CosineAnnealingLR smoothly decays to near zero following a cosine curve; ReduceLROnPlateau decreases LR when validation loss stops improving. Schedulers are stepped after the optimizer, typically once per epoch.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Decay LR by 0.5 every 2 epochs
scheduler = optim.lr_scheduler.StepLR(
    optimizer, step_size=2, gamma=0.5
)

for epoch in range(6):
    # ... training code ...
    scheduler.step()
    print(f'Epoch {epoch}: LR={scheduler.get_last_lr()[0]}')
# LR: 0.01, 0.01, 0.005, 0.005, 0.0025, 0.0025

Saving Checkpoints During Training

Training can take hours or days — saving checkpoints periodically protects against crashes and lets you resume from the best point. A good checkpoint stores the model state dict, optimizer state dict (contains momentum terms), epoch number, and best validation metric. Restoring the optimizer state allows training to resume exactly where it left off, including momentum buffers that affect subsequent updates.

import torch

def save_checkpoint(model, optimizer, epoch, loss, path):
    torch.save({
        'epoch': epoch,
        'model_state_dict': model.state_dict(),
        'optimizer_state_dict': optimizer.state_dict(),
        'loss': loss
    }, path)

def load_checkpoint(model, optimizer, path):
    ckpt = torch.load(path)
    model.load_state_dict(ckpt['model_state_dict'])
    optimizer.load_state_dict(ckpt['optimizer_state_dict'])
    return ckpt['epoch'], ckpt['loss']

Complete Training Loop: Putting It Together

A production-quality training loop combines all the pieces: DataLoader for batching, the four-step update per batch, a validation pass per epoch, learning rate scheduling, loss and accuracy tracking, and checkpoint saving. The code below shows the full pattern with a simple classification model. Adapting this template to any supervised learning problem requires changing only the model, dataset, and loss function.

import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader

X_tr = torch.randn(160, 4); y_tr = torch.randint(0, 3, (160,))
X_val = torch.randn(40, 4); y_val = torch.randint(0, 3, (40,))
tr_loader = DataLoader(TensorDataset(X_tr, y_tr), 32, shuffle=True)
val_loader = DataLoader(TensorDataset(X_val, y_val), 32)

model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

for epoch in range(5):
    model.train()
    for Xb, yb in tr_loader:
        optimizer.zero_grad()
        loss = criterion(model(Xb), yb)
        loss.backward(); optimizer.step()
    model.eval()
    with torch.no_grad():
        val_loss = sum(criterion(model(Xb), yb).item()
                       for Xb, yb in val_loader) / len(val_loader)
    print(f'Epoch {epoch}: val_loss={val_loss:.3f}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the four-step training loop (zero_grad, forward, backward, step) is the foundation of all PyTorch training, DataLoader handles batching and shuffling automatically, and CrossEntropyLoss with logits is the standard choice for classification. Next up we explore learning rate scheduling and the impact of the learning rate on training dynamics.

Häufig gestellte Fragen

Ist die Lektion „Trainingsschleife: Loss, Optimizer und Epochen“ kostenlos?

Ja — der vollständige Text von „Trainingsschleife: Loss, Optimizer und Epochen“ ist hier im Web kostenlos zu lesen. Um sie interaktiv zu üben (integrierter Code-Editor und 24/7 KI-Tutor) und den Rest des Machine Learning Academy-Kurses freizuschalten, upgrade auf CoddyKit PRO. Der Machine Learning Academy-Kurs umfasst insgesamt 4 Lektionen.

Was lerne ich in „Trainingsschleife: Loss, Optimizer und Epochen“?

Lernende schreiben die PyTorch-Trainingsschleife: zero_grad, Forward-Pass, Berechnung von CrossEntropyLoss, Backward-Pass und optimizer.step, und verfolgen Loss und Genauigkeit über mehrere Epochen. Du übst Machine Learning Academy mit praktischem Code, den du direkt im Browser ausführst, und ein 24/7 KI-Tutor beantwortet deine Fragen während du die Lektion bearbeitest.

Brauche ich Erfahrung, um Machine Learning Academy zu starten?

Keine Vorkenntnisse erforderlich. Machine Learning Academy auf CoddyKit ist für Anfänger bis fortgeschrittene Lernende strukturiert, sodass du hier starten oder von Anfang an beginnen und in deinem eigenen Tempo voranschreiten kannst. Dies ist Lektion 4 von 4.

Wie lange dauert die Lektion „Trainingsschleife: Loss, Optimizer und Epochen“?

Die meisten CoddyKit-Lektionen dauern etwa 5–10 Minuten. Jede ist kompakt und interaktiv, sodass du stetig Fortschritte machst und genau dort weitermachst, wo du aufgehört hast – im Web und in der App.

Kann ich in dieser Machine Learning Academy-Lektion Code schreiben und ausführen?

Ja. Jede Machine Learning Academy-Lektion enthält einen integrierten Code-Editor, sodass du echten Code direkt in deinem Browser schreibst und ausführst und sofort KI-Feedback erhältst — ohne lokale Einrichtung erforderlich.

Alle Lektionen in diesem Kurs

  1. PyTorch-Tensoren: Erstellen, Operationen und GPU-Übertragung
  2. Autograd: Automatische Differentiation für Backpropagation
  3. Ein Feedforward-Netzwerk mit nn.Module erstellen
  4. Trainingsschleife: Loss, Optimizer und Epochen
← Zurück zu Machine Learning Academy