Machine Learning Academy · 课时

训练循环：损失、优化器与训练轮次

您将编写 PyTorch 训练循环：执行 zero_grad、前向传播、计算 CrossEntropyLoss、反向传播和 optimizer.step，并跟踪各训练轮次的损失与准确率。

第 4 / 4 课13 个步骤

训练循环：损失、优化器与训练轮次 是 CoddyKit 上的免费 Machine Learning Academy 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Machine Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Machine Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

The Four Steps of Every Training Loop

The PyTorch training loop has four mandatory steps that repeat for every batch: (1) zero gradients, (2) forward pass, (3) backward pass, and (4) optimizer step. Skipping or reordering these steps produces wrong results silently — gradients accumulate, parameters update incorrectly, or memory leaks. Internalising this pattern is the most important habit for training neural networks with PyTorch.

import torch
import torch.nn as nn
import torch.optim as optim

model = nn.Linear(2, 1)
optimizer = optim.SGD(model.parameters(), lr=0.01)
criterion = nn.MSELoss()

X = torch.randn(20, 2)
y = torch.randn(20, 1)

for step in range(1):
    optimizer.zero_grad()           # (1) zero grads
    y_pred = model(X)               # (2) forward
    loss = criterion(y_pred, y)     # (2) loss
    loss.backward()                 # (3) backward
    optimizer.step()                # (4) update
    print('Loss:', loss.item())

Loss Functions: Choosing the Right Criterion

The loss function measures how wrong the model's predictions are. PyTorch's nn module provides many: nn.MSELoss for regression (mean squared error), nn.CrossEntropyLoss for multi-class classification (combines log-softmax and NLL), and nn.BCEWithLogitsLoss for binary classification (combines sigmoid and binary cross-entropy). Using the wrong loss for your task is a common beginner mistake that prevents learning.

import torch
import torch.nn as nn

# Regression
mse = nn.MSELoss()
y_pred = torch.tensor([2.5, 3.0])
y_true = torch.tensor([2.0, 3.5])
print('MSE:', mse(y_pred, y_true).item())

# Multi-class: logits (raw scores), not softmax
ce = nn.CrossEntropyLoss()
logits = torch.tensor([[2.0, 0.5, 1.0]])
labels = torch.tensor([0])
print('CE:', ce(logits, labels).item())

Optimizers: SGD, Adam, and AdamW

An optimizer uses computed gradients to update model parameters. SGD (Stochastic Gradient Descent) is the classic optimizer; adding momentum accelerates convergence. Adam adapts the learning rate per parameter using first and second moment estimates, converging faster in practice. AdamW adds proper weight decay (L2 regularisation) and is the default choice for transformer models. All optimizers are in torch.optim.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)

# Stochastic Gradient Descent with momentum
sgd = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)

# Adam: adaptive learning rate
adam = optim.Adam(model.parameters(), lr=0.001,
                  betas=(0.9, 0.999))

# AdamW: Adam + proper weight decay
adamw = optim.AdamW(model.parameters(), lr=0.001,
                    weight_decay=0.01)

Iterating Over Epochs and Batches

Training typically runs for many epochs — complete passes through the training dataset. Within each epoch you iterate over batches (subsets of the data). Using a DataLoader handles shuffling and batching automatically. Tracking average loss per epoch lets you monitor whether the model is converging. Printing every epoch (or every N batches) gives you visibility into training progress.

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn as nn
import torch.optim as optim

X = torch.randn(200, 4)
y = torch.randn(200, 1)
dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

model = nn.Linear(4, 1)
optimizer = optim.Adam(model.parameters())
criterion = nn.MSELoss()

for epoch in range(3):
    total_loss = 0
    for X_batch, y_batch in loader:
        optimizer.zero_grad()
        pred = model(X_batch)
        loss = criterion(pred, y_batch)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f'Epoch {epoch}: avg_loss={total_loss/len(loader):.4f}')

DataLoader: Batching and Shuffling Data

DataLoader wraps a Dataset and provides an iterator that yields batches. Key parameters: batch_size controls how many samples per gradient update; shuffle=True randomises order each epoch (crucial for training); num_workers enables parallel data loading. For custom datasets, subclass torch.utils.data.Dataset and implement __len__ and __getitem__.

import torch
from torch.utils.data import Dataset, DataLoader

class MyDataset(Dataset):
    def __init__(self, X, y):
        self.X = X
        self.y = y

    def __len__(self):
        return len(self.X)

    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

X = torch.randn(100, 5)
y = torch.randint(0, 3, (100,))
dataset = MyDataset(X, y)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

X_batch, y_batch = next(iter(loader))
print(X_batch.shape)   # torch.Size([16, 5])

CrossEntropyLoss: Logits and Class Indices

nn.CrossEntropyLoss expects raw logits (unnormalised scores), not softmax probabilities. It internally applies log-softmax and computes negative log-likelihood. Target labels should be integer class indices (not one-hot vectors). This is the correct loss for multi-class classification and is more numerically stable than manually applying softmax then NLLLoss. The model's final layer should NOT have a softmax activation when using this loss.

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss()

# Batch of 4 samples, 3 classes
logits = torch.tensor([
    [2.1, 0.5, 0.3],
    [0.1, 3.0, 0.2],
    [0.8, 0.9, 2.0],
    [1.5, 0.1, 0.4]
])

# True labels as class indices (not one-hot)
labels = torch.tensor([0, 1, 2, 0])

loss = criterion(logits, labels)
print('Loss:', loss.item())

Tracking Accuracy During Training

Loss decreasing confirms the model is learning, but accuracy tells you whether predictions are actually correct. For multi-class classification, convert logits to predicted class with torch.argmax along the class dimension, then compare with true labels. Tracking both training loss and training accuracy, and optionally validation accuracy, gives a complete picture of learning dynamics.

import torch

def compute_accuracy(logits, labels):
    preds = torch.argmax(logits, dim=1)
    correct = (preds == labels).sum().item()
    return correct / len(labels)

logits = torch.tensor([
    [2.0, 0.1, 0.3],
    [0.1, 0.2, 3.5],
    [1.0, 0.5, 0.2]
])
labels = torch.tensor([0, 2, 0])

print('Accuracy:', compute_accuracy(logits, labels))
# 1.0 (all three correct)

Validation Loop: Evaluating Without Updating

After each training epoch, run a validation loop to assess performance on unseen data. Wrap it with torch.no_grad() to disable gradient computation and call model.eval() to deactivate Dropout and BatchNorm stochasticity. Comparing training loss vs validation loss is the primary tool for detecting overfitting — when validation loss rises while training loss continues to fall, the model has overfit the training set.

import torch

def validate(model, val_loader, criterion):
    model.eval()
    total_loss = 0
    with torch.no_grad():
        for X_batch, y_batch in val_loader:
            logits = model(X_batch)
            loss = criterion(logits, y_batch)
            total_loss += loss.item()
    model.train()   # restore training mode
    return total_loss / len(val_loader)

# Usage inside training loop:
# val_loss = validate(model, val_loader, criterion)
# print(f'Val loss: {val_loss:.4f}')

Learning Rate Scheduling

A fixed learning rate often leads to slow convergence or oscillation near the minimum. Learning rate schedulers automatically adjust the LR during training. StepLR reduces LR by a factor every N epochs; CosineAnnealingLR smoothly decays to near zero following a cosine curve; ReduceLROnPlateau decreases LR when validation loss stops improving. Schedulers are stepped after the optimizer, typically once per epoch.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Decay LR by 0.5 every 2 epochs
scheduler = optim.lr_scheduler.StepLR(
    optimizer, step_size=2, gamma=0.5
)

for epoch in range(6):
    # ... training code ...
    scheduler.step()
    print(f'Epoch {epoch}: LR={scheduler.get_last_lr()[0]}')
# LR: 0.01, 0.01, 0.005, 0.005, 0.0025, 0.0025

Saving Checkpoints During Training

Training can take hours or days — saving checkpoints periodically protects against crashes and lets you resume from the best point. A good checkpoint stores the model state dict, optimizer state dict (contains momentum terms), epoch number, and best validation metric. Restoring the optimizer state allows training to resume exactly where it left off, including momentum buffers that affect subsequent updates.

import torch

def save_checkpoint(model, optimizer, epoch, loss, path):
    torch.save({
        'epoch': epoch,
        'model_state_dict': model.state_dict(),
        'optimizer_state_dict': optimizer.state_dict(),
        'loss': loss
    }, path)

def load_checkpoint(model, optimizer, path):
    ckpt = torch.load(path)
    model.load_state_dict(ckpt['model_state_dict'])
    optimizer.load_state_dict(ckpt['optimizer_state_dict'])
    return ckpt['epoch'], ckpt['loss']

Complete Training Loop: Putting It Together

A production-quality training loop combines all the pieces: DataLoader for batching, the four-step update per batch, a validation pass per epoch, learning rate scheduling, loss and accuracy tracking, and checkpoint saving. The code below shows the full pattern with a simple classification model. Adapting this template to any supervised learning problem requires changing only the model, dataset, and loss function.

import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader

X_tr = torch.randn(160, 4); y_tr = torch.randint(0, 3, (160,))
X_val = torch.randn(40, 4); y_val = torch.randint(0, 3, (40,))
tr_loader = DataLoader(TensorDataset(X_tr, y_tr), 32, shuffle=True)
val_loader = DataLoader(TensorDataset(X_val, y_val), 32)

model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

for epoch in range(5):
    model.train()
    for Xb, yb in tr_loader:
        optimizer.zero_grad()
        loss = criterion(model(Xb), yb)
        loss.backward(); optimizer.step()
    model.eval()
    with torch.no_grad():
        val_loss = sum(criterion(model(Xb), yb).item()
                       for Xb, yb in val_loader) / len(val_loader)
    print(f'Epoch {epoch}: val_loss={val_loss:.3f}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

在交互式课程中回答

Lesson Recap

In this lesson you learned: the four-step training loop (zero_grad, forward, backward, step) is the foundation of all PyTorch training, DataLoader handles batching and shuffling automatically, and CrossEntropyLoss with logits is the standard choice for classification. Next up we explore learning rate scheduling and the impact of the learning rate on training dynamics.

免费开始

用 AI 导师学习 Python — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程: 30
课程: 120

免费开始学习查看完整课程

常见问题解答

「训练循环：损失、优化器与训练轮次」课时是免费的吗？

是的 — 「训练循环：损失、优化器与训练轮次」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Machine Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Machine Learning Academy 课程共包含 4 节课。

「训练循环：损失、优化器与训练轮次」这节课中我会学到什么？

您将编写 PyTorch 训练循环：执行 zero_grad、前向传播、计算 CrossEntropyLoss、反向传播和 optimizer.step，并跟踪各训练轮次的损失与准确率。你通过在浏览器中直接运行的动手代码来练习 Machine Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Machine Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Machine Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「训练循环：损失、优化器与训练轮次」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Machine Learning Academy 课中编写并运行代码吗？

能。每节 Machine Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Machine Learning Academy