0Pricing
Machine Learning Academy · Lekcja

Pętla trenowania: funkcja straty, optymalizator i epoki

Uczą się Państwo pisać pętlę trenowania w PyTorch: zero_grad, przebieg w przód, obliczanie CrossEntropyLoss, backward, optimizer.step oraz śledzenie straty i accuracy w kolejnych epokach.

Pętla trenowania: funkcja straty, optymalizator i epoki to bezpłatna lekcja Machine Learning Academy na CoddyKit. To lekcja 4 z 4. Możesz przeczytać całą lekcję poniżej za darmo — a potem ćwiczyć ją interaktywnie w przeglądarce z wbudowanym edytorem kodu i tutorem AI dostępnym 24/7. To część ścieżki edukacyjnej Machine Learning Academy, a Twój postęp synchronizuje się między webem a aplikacją CoddyKit. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Części tej lekcji nie zostały jeszcze przetłumaczone i są wyświetlane po angielsku.

The Four Steps of Every Training Loop

The PyTorch training loop has four mandatory steps that repeat for every batch: (1) zero gradients, (2) forward pass, (3) backward pass, and (4) optimizer step. Skipping or reordering these steps produces wrong results silently — gradients accumulate, parameters update incorrectly, or memory leaks. Internalising this pattern is the most important habit for training neural networks with PyTorch.

import torch
import torch.nn as nn
import torch.optim as optim

model = nn.Linear(2, 1)
optimizer = optim.SGD(model.parameters(), lr=0.01)
criterion = nn.MSELoss()

X = torch.randn(20, 2)
y = torch.randn(20, 1)

for step in range(1):
    optimizer.zero_grad()           # (1) zero grads
    y_pred = model(X)               # (2) forward
    loss = criterion(y_pred, y)     # (2) loss
    loss.backward()                 # (3) backward
    optimizer.step()                # (4) update
    print('Loss:', loss.item())

Loss Functions: Choosing the Right Criterion

The loss function measures how wrong the model's predictions are. PyTorch's nn module provides many: nn.MSELoss for regression (mean squared error), nn.CrossEntropyLoss for multi-class classification (combines log-softmax and NLL), and nn.BCEWithLogitsLoss for binary classification (combines sigmoid and binary cross-entropy). Using the wrong loss for your task is a common beginner mistake that prevents learning.

import torch
import torch.nn as nn

# Regression
mse = nn.MSELoss()
y_pred = torch.tensor([2.5, 3.0])
y_true = torch.tensor([2.0, 3.5])
print('MSE:', mse(y_pred, y_true).item())

# Multi-class: logits (raw scores), not softmax
ce = nn.CrossEntropyLoss()
logits = torch.tensor([[2.0, 0.5, 1.0]])
labels = torch.tensor([0])
print('CE:', ce(logits, labels).item())

Optimizers: SGD, Adam, and AdamW

An optimizer uses computed gradients to update model parameters. SGD (Stochastic Gradient Descent) is the classic optimizer; adding momentum accelerates convergence. Adam adapts the learning rate per parameter using first and second moment estimates, converging faster in practice. AdamW adds proper weight decay (L2 regularisation) and is the default choice for transformer models. All optimizers are in torch.optim.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)

# Stochastic Gradient Descent with momentum
sgd = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)

# Adam: adaptive learning rate
adam = optim.Adam(model.parameters(), lr=0.001,
                  betas=(0.9, 0.999))

# AdamW: Adam + proper weight decay
adamw = optim.AdamW(model.parameters(), lr=0.001,
                    weight_decay=0.01)

Iterating Over Epochs and Batches

Training typically runs for many epochs — complete passes through the training dataset. Within each epoch you iterate over batches (subsets of the data). Using a DataLoader handles shuffling and batching automatically. Tracking average loss per epoch lets you monitor whether the model is converging. Printing every epoch (or every N batches) gives you visibility into training progress.

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn as nn
import torch.optim as optim

X = torch.randn(200, 4)
y = torch.randn(200, 1)
dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

model = nn.Linear(4, 1)
optimizer = optim.Adam(model.parameters())
criterion = nn.MSELoss()

for epoch in range(3):
    total_loss = 0
    for X_batch, y_batch in loader:
        optimizer.zero_grad()
        pred = model(X_batch)
        loss = criterion(pred, y_batch)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f'Epoch {epoch}: avg_loss={total_loss/len(loader):.4f}')

DataLoader: Batching and Shuffling Data

DataLoader wraps a Dataset and provides an iterator that yields batches. Key parameters: batch_size controls how many samples per gradient update; shuffle=True randomises order each epoch (crucial for training); num_workers enables parallel data loading. For custom datasets, subclass torch.utils.data.Dataset and implement __len__ and __getitem__.

import torch
from torch.utils.data import Dataset, DataLoader

class MyDataset(Dataset):
    def __init__(self, X, y):
        self.X = X
        self.y = y

    def __len__(self):
        return len(self.X)

    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

X = torch.randn(100, 5)
y = torch.randint(0, 3, (100,))
dataset = MyDataset(X, y)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

X_batch, y_batch = next(iter(loader))
print(X_batch.shape)   # torch.Size([16, 5])

CrossEntropyLoss: Logits and Class Indices

nn.CrossEntropyLoss expects raw logits (unnormalised scores), not softmax probabilities. It internally applies log-softmax and computes negative log-likelihood. Target labels should be integer class indices (not one-hot vectors). This is the correct loss for multi-class classification and is more numerically stable than manually applying softmax then NLLLoss. The model's final layer should NOT have a softmax activation when using this loss.

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss()

# Batch of 4 samples, 3 classes
logits = torch.tensor([
    [2.1, 0.5, 0.3],
    [0.1, 3.0, 0.2],
    [0.8, 0.9, 2.0],
    [1.5, 0.1, 0.4]
])

# True labels as class indices (not one-hot)
labels = torch.tensor([0, 1, 2, 0])

loss = criterion(logits, labels)
print('Loss:', loss.item())

Tracking Accuracy During Training

Loss decreasing confirms the model is learning, but accuracy tells you whether predictions are actually correct. For multi-class classification, convert logits to predicted class with torch.argmax along the class dimension, then compare with true labels. Tracking both training loss and training accuracy, and optionally validation accuracy, gives a complete picture of learning dynamics.

import torch

def compute_accuracy(logits, labels):
    preds = torch.argmax(logits, dim=1)
    correct = (preds == labels).sum().item()
    return correct / len(labels)

logits = torch.tensor([
    [2.0, 0.1, 0.3],
    [0.1, 0.2, 3.5],
    [1.0, 0.5, 0.2]
])
labels = torch.tensor([0, 2, 0])

print('Accuracy:', compute_accuracy(logits, labels))
# 1.0 (all three correct)

Validation Loop: Evaluating Without Updating

After each training epoch, run a validation loop to assess performance on unseen data. Wrap it with torch.no_grad() to disable gradient computation and call model.eval() to deactivate Dropout and BatchNorm stochasticity. Comparing training loss vs validation loss is the primary tool for detecting overfitting — when validation loss rises while training loss continues to fall, the model has overfit the training set.

import torch

def validate(model, val_loader, criterion):
    model.eval()
    total_loss = 0
    with torch.no_grad():
        for X_batch, y_batch in val_loader:
            logits = model(X_batch)
            loss = criterion(logits, y_batch)
            total_loss += loss.item()
    model.train()   # restore training mode
    return total_loss / len(val_loader)

# Usage inside training loop:
# val_loss = validate(model, val_loader, criterion)
# print(f'Val loss: {val_loss:.4f}')

Learning Rate Scheduling

A fixed learning rate often leads to slow convergence or oscillation near the minimum. Learning rate schedulers automatically adjust the LR during training. StepLR reduces LR by a factor every N epochs; CosineAnnealingLR smoothly decays to near zero following a cosine curve; ReduceLROnPlateau decreases LR when validation loss stops improving. Schedulers are stepped after the optimizer, typically once per epoch.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Decay LR by 0.5 every 2 epochs
scheduler = optim.lr_scheduler.StepLR(
    optimizer, step_size=2, gamma=0.5
)

for epoch in range(6):
    # ... training code ...
    scheduler.step()
    print(f'Epoch {epoch}: LR={scheduler.get_last_lr()[0]}')
# LR: 0.01, 0.01, 0.005, 0.005, 0.0025, 0.0025

Saving Checkpoints During Training

Training can take hours or days — saving checkpoints periodically protects against crashes and lets you resume from the best point. A good checkpoint stores the model state dict, optimizer state dict (contains momentum terms), epoch number, and best validation metric. Restoring the optimizer state allows training to resume exactly where it left off, including momentum buffers that affect subsequent updates.

import torch

def save_checkpoint(model, optimizer, epoch, loss, path):
    torch.save({
        'epoch': epoch,
        'model_state_dict': model.state_dict(),
        'optimizer_state_dict': optimizer.state_dict(),
        'loss': loss
    }, path)

def load_checkpoint(model, optimizer, path):
    ckpt = torch.load(path)
    model.load_state_dict(ckpt['model_state_dict'])
    optimizer.load_state_dict(ckpt['optimizer_state_dict'])
    return ckpt['epoch'], ckpt['loss']

Complete Training Loop: Putting It Together

A production-quality training loop combines all the pieces: DataLoader for batching, the four-step update per batch, a validation pass per epoch, learning rate scheduling, loss and accuracy tracking, and checkpoint saving. The code below shows the full pattern with a simple classification model. Adapting this template to any supervised learning problem requires changing only the model, dataset, and loss function.

import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader

X_tr = torch.randn(160, 4); y_tr = torch.randint(0, 3, (160,))
X_val = torch.randn(40, 4); y_val = torch.randint(0, 3, (40,))
tr_loader = DataLoader(TensorDataset(X_tr, y_tr), 32, shuffle=True)
val_loader = DataLoader(TensorDataset(X_val, y_val), 32)

model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

for epoch in range(5):
    model.train()
    for Xb, yb in tr_loader:
        optimizer.zero_grad()
        loss = criterion(model(Xb), yb)
        loss.backward(); optimizer.step()
    model.eval()
    with torch.no_grad():
        val_loss = sum(criterion(model(Xb), yb).item()
                       for Xb, yb in val_loader) / len(val_loader)
    print(f'Epoch {epoch}: val_loss={val_loss:.3f}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: the four-step training loop (zero_grad, forward, backward, step) is the foundation of all PyTorch training, DataLoader handles batching and shuffling automatically, and CrossEntropyLoss with logits is the standard choice for classification. Next up we explore learning rate scheduling and the impact of the learning rate on training dynamics.

Często zadawane pytania

Czy lekcja „Pętla trenowania: funkcja straty, optymalizator i epoki” jest bezpłatna?

Tak — pełny tekst „Pętla trenowania: funkcja straty, optymalizator i epoki” jest dostępny za darmo tutaj w sieci. Aby ćwiczyć ją interaktywnie (wbudowany edytor kodu i tutor AI dostępny 24/7) i odblokować resztę kursu Machine Learning Academy, przejdź na CoddyKit PRO. Kurs Machine Learning Academy zawiera 4 lekcji w sumie.

Co nauczysz się w „Pętla trenowania: funkcja straty, optymalizator i epoki”?

Uczą się Państwo pisać pętlę trenowania w PyTorch: zero_grad, przebieg w przód, obliczanie CrossEntropyLoss, backward, optimizer.step oraz śledzenie straty i accuracy w kolejnych epokach. Ćwiczysz Machine Learning Academy z praktycznym kodem, który uruchamiasz bezpośrednio w przeglądarce, a tutor AI dostępny 24/7 odpowiada na Twoje pytania podczas pracy nad lekcją.

Czy potrzebuję doświadczenia, aby zacząć Machine Learning Academy?

Nie wymagamy żadnego doświadczenia. Machine Learning Academy w CoddyKit jest strukturyzowany dla początkujących i zaawansowanych użytkowników, więc możesz zacząć tutaj lub od początku i uczyć się w swoim tempie. To lekcja 4 z 4.

Ile czasu zajmuje lekcja „Pętla trenowania: funkcja straty, optymalizator i epoki”?

Większość lekcji CoddyKit trwa około 5–10 minut. Każda lekcja to mały, interaktywny krok, dzięki czemu robisz systematyczne postępy i zawsze wracasz dokładnie do tego samego miejsca — na webie i w aplikacji.

Czy mogę pisać i uruchamiać kod w tej lekcji Machine Learning Academy?

Tak. Każda lekcja Machine Learning Academy zawiera wbudowany edytor kodu, więc piszesz i uruchamiasz prawdziwy kod bezpośrednio w przeglądarce i od razu otrzymujesz sprzężenie zwrotne od AI — bez konfiguracji na komputerze.

Wszystkie lekcje w tym kursie

  1. Tensory PyTorch: tworzenie, operacje i przenoszenie na GPU
  2. Autograd: automatyczne różniczkowanie na potrzeby propagacji wstecznej
  3. Tworzenie sieci jednokierunkowej za pomocą nn.Module
  4. Pętla trenowania: funkcja straty, optymalizator i epoki
← Powrót do Machine Learning Academy