Machine Learning Academy · 강의

학습 반복: 손실, 옵티마이저 및 에포크

학습자는 PyTorch 학습 반복을 작성합니다. zero_grad, 순전파, CrossEntropyLoss 계산, 역전파 및 optimizer.step을 수행하고 에포크별 손실과 정확도를 추적합니다.

레슨 4/413개 단계

학습 반복: 손실, 옵티마이저 및 에포크은(는) CoddyKit의 무료 Machine Learning Academy 강의입니다. 이것은 4개 중 4번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Machine Learning Academy 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Machine Learning Academy 강의에는 총 4개의 강의가 포함되어 있습니다.

이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.

The Four Steps of Every Training Loop

The PyTorch training loop has four mandatory steps that repeat for every batch: (1) zero gradients, (2) forward pass, (3) backward pass, and (4) optimizer step. Skipping or reordering these steps produces wrong results silently — gradients accumulate, parameters update incorrectly, or memory leaks. Internalising this pattern is the most important habit for training neural networks with PyTorch.

import torch
import torch.nn as nn
import torch.optim as optim

model = nn.Linear(2, 1)
optimizer = optim.SGD(model.parameters(), lr=0.01)
criterion = nn.MSELoss()

X = torch.randn(20, 2)
y = torch.randn(20, 1)

for step in range(1):
    optimizer.zero_grad()           # (1) zero grads
    y_pred = model(X)               # (2) forward
    loss = criterion(y_pred, y)     # (2) loss
    loss.backward()                 # (3) backward
    optimizer.step()                # (4) update
    print('Loss:', loss.item())

Loss Functions: Choosing the Right Criterion

The loss function measures how wrong the model's predictions are. PyTorch's nn module provides many: nn.MSELoss for regression (mean squared error), nn.CrossEntropyLoss for multi-class classification (combines log-softmax and NLL), and nn.BCEWithLogitsLoss for binary classification (combines sigmoid and binary cross-entropy). Using the wrong loss for your task is a common beginner mistake that prevents learning.

import torch
import torch.nn as nn

# Regression
mse = nn.MSELoss()
y_pred = torch.tensor([2.5, 3.0])
y_true = torch.tensor([2.0, 3.5])
print('MSE:', mse(y_pred, y_true).item())

# Multi-class: logits (raw scores), not softmax
ce = nn.CrossEntropyLoss()
logits = torch.tensor([[2.0, 0.5, 1.0]])
labels = torch.tensor([0])
print('CE:', ce(logits, labels).item())

Optimizers: SGD, Adam, and AdamW

An optimizer uses computed gradients to update model parameters. SGD (Stochastic Gradient Descent) is the classic optimizer; adding momentum accelerates convergence. Adam adapts the learning rate per parameter using first and second moment estimates, converging faster in practice. AdamW adds proper weight decay (L2 regularisation) and is the default choice for transformer models. All optimizers are in torch.optim.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)

# Stochastic Gradient Descent with momentum
sgd = optim.SGD(model.parameters(), lr=0.01, momentum=0.9)

# Adam: adaptive learning rate
adam = optim.Adam(model.parameters(), lr=0.001,
                  betas=(0.9, 0.999))

# AdamW: Adam + proper weight decay
adamw = optim.AdamW(model.parameters(), lr=0.001,
                    weight_decay=0.01)

Iterating Over Epochs and Batches

Training typically runs for many epochs — complete passes through the training dataset. Within each epoch you iterate over batches (subsets of the data). Using a DataLoader handles shuffling and batching automatically. Tracking average loss per epoch lets you monitor whether the model is converging. Printing every epoch (or every N batches) gives you visibility into training progress.

import torch
from torch.utils.data import TensorDataset, DataLoader
import torch.nn as nn
import torch.optim as optim

X = torch.randn(200, 4)
y = torch.randn(200, 1)
dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=32, shuffle=True)

model = nn.Linear(4, 1)
optimizer = optim.Adam(model.parameters())
criterion = nn.MSELoss()

for epoch in range(3):
    total_loss = 0
    for X_batch, y_batch in loader:
        optimizer.zero_grad()
        pred = model(X_batch)
        loss = criterion(pred, y_batch)
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f'Epoch {epoch}: avg_loss={total_loss/len(loader):.4f}')

DataLoader: Batching and Shuffling Data

DataLoader wraps a Dataset and provides an iterator that yields batches. Key parameters: batch_size controls how many samples per gradient update; shuffle=True randomises order each epoch (crucial for training); num_workers enables parallel data loading. For custom datasets, subclass torch.utils.data.Dataset and implement __len__ and __getitem__.

import torch
from torch.utils.data import Dataset, DataLoader

class MyDataset(Dataset):
    def __init__(self, X, y):
        self.X = X
        self.y = y

    def __len__(self):
        return len(self.X)

    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

X = torch.randn(100, 5)
y = torch.randint(0, 3, (100,))
dataset = MyDataset(X, y)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

X_batch, y_batch = next(iter(loader))
print(X_batch.shape)   # torch.Size([16, 5])

CrossEntropyLoss: Logits and Class Indices

nn.CrossEntropyLoss expects raw logits (unnormalised scores), not softmax probabilities. It internally applies log-softmax and computes negative log-likelihood. Target labels should be integer class indices (not one-hot vectors). This is the correct loss for multi-class classification and is more numerically stable than manually applying softmax then NLLLoss. The model's final layer should NOT have a softmax activation when using this loss.

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss()

# Batch of 4 samples, 3 classes
logits = torch.tensor([
    [2.1, 0.5, 0.3],
    [0.1, 3.0, 0.2],
    [0.8, 0.9, 2.0],
    [1.5, 0.1, 0.4]
])

# True labels as class indices (not one-hot)
labels = torch.tensor([0, 1, 2, 0])

loss = criterion(logits, labels)
print('Loss:', loss.item())

Tracking Accuracy During Training

Loss decreasing confirms the model is learning, but accuracy tells you whether predictions are actually correct. For multi-class classification, convert logits to predicted class with torch.argmax along the class dimension, then compare with true labels. Tracking both training loss and training accuracy, and optionally validation accuracy, gives a complete picture of learning dynamics.

import torch

def compute_accuracy(logits, labels):
    preds = torch.argmax(logits, dim=1)
    correct = (preds == labels).sum().item()
    return correct / len(labels)

logits = torch.tensor([
    [2.0, 0.1, 0.3],
    [0.1, 0.2, 3.5],
    [1.0, 0.5, 0.2]
])
labels = torch.tensor([0, 2, 0])

print('Accuracy:', compute_accuracy(logits, labels))
# 1.0 (all three correct)

Validation Loop: Evaluating Without Updating

After each training epoch, run a validation loop to assess performance on unseen data. Wrap it with torch.no_grad() to disable gradient computation and call model.eval() to deactivate Dropout and BatchNorm stochasticity. Comparing training loss vs validation loss is the primary tool for detecting overfitting — when validation loss rises while training loss continues to fall, the model has overfit the training set.

import torch

def validate(model, val_loader, criterion):
    model.eval()
    total_loss = 0
    with torch.no_grad():
        for X_batch, y_batch in val_loader:
            logits = model(X_batch)
            loss = criterion(logits, y_batch)
            total_loss += loss.item()
    model.train()   # restore training mode
    return total_loss / len(val_loader)

# Usage inside training loop:
# val_loss = validate(model, val_loader, criterion)
# print(f'Val loss: {val_loss:.4f}')

Learning Rate Scheduling

A fixed learning rate often leads to slow convergence or oscillation near the minimum. Learning rate schedulers automatically adjust the LR during training. StepLR reduces LR by a factor every N epochs; CosineAnnealingLR smoothly decays to near zero following a cosine curve; ReduceLROnPlateau decreases LR when validation loss stops improving. Schedulers are stepped after the optimizer, typically once per epoch.

import torch.optim as optim
import torch.nn as nn

model = nn.Linear(4, 2)
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Decay LR by 0.5 every 2 epochs
scheduler = optim.lr_scheduler.StepLR(
    optimizer, step_size=2, gamma=0.5
)

for epoch in range(6):
    # ... training code ...
    scheduler.step()
    print(f'Epoch {epoch}: LR={scheduler.get_last_lr()[0]}')
# LR: 0.01, 0.01, 0.005, 0.005, 0.0025, 0.0025

Saving Checkpoints During Training

Training can take hours or days — saving checkpoints periodically protects against crashes and lets you resume from the best point. A good checkpoint stores the model state dict, optimizer state dict (contains momentum terms), epoch number, and best validation metric. Restoring the optimizer state allows training to resume exactly where it left off, including momentum buffers that affect subsequent updates.

import torch

def save_checkpoint(model, optimizer, epoch, loss, path):
    torch.save({
        'epoch': epoch,
        'model_state_dict': model.state_dict(),
        'optimizer_state_dict': optimizer.state_dict(),
        'loss': loss
    }, path)

def load_checkpoint(model, optimizer, path):
    ckpt = torch.load(path)
    model.load_state_dict(ckpt['model_state_dict'])
    optimizer.load_state_dict(ckpt['optimizer_state_dict'])
    return ckpt['epoch'], ckpt['loss']

Complete Training Loop: Putting It Together

A production-quality training loop combines all the pieces: DataLoader for batching, the four-step update per batch, a validation pass per epoch, learning rate scheduling, loss and accuracy tracking, and checkpoint saving. The code below shows the full pattern with a simple classification model. Adapting this template to any supervised learning problem requires changing only the model, dataset, and loss function.

import torch, torch.nn as nn, torch.optim as optim
from torch.utils.data import TensorDataset, DataLoader

X_tr = torch.randn(160, 4); y_tr = torch.randint(0, 3, (160,))
X_val = torch.randn(40, 4); y_val = torch.randint(0, 3, (40,))
tr_loader = DataLoader(TensorDataset(X_tr, y_tr), 32, shuffle=True)
val_loader = DataLoader(TensorDataset(X_val, y_val), 32)

model = nn.Sequential(nn.Linear(4, 16), nn.ReLU(), nn.Linear(16, 3))
optimizer = optim.Adam(model.parameters())
criterion = nn.CrossEntropyLoss()

for epoch in range(5):
    model.train()
    for Xb, yb in tr_loader:
        optimizer.zero_grad()
        loss = criterion(model(Xb), yb)
        loss.backward(); optimizer.step()
    model.eval()
    with torch.no_grad():
        val_loss = sum(criterion(model(Xb), yb).item()
                       for Xb, yb in val_loader) / len(val_loader)
    print(f'Epoch {epoch}: val_loss={val_loss:.3f}')

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

인터랙티브 레슨에서 답해보세요

Lesson Recap

In this lesson you learned: the four-step training loop (zero_grad, forward, backward, step) is the foundation of all PyTorch training, DataLoader handles batching and shuffling automatically, and CrossEntropyLoss with logits is the standard choice for classification. Next up we explore learning rate scheduling and the impact of the learning rate on training dynamics.

무료로 시작

AI 튜터와 함께 Python을(를) 배우세요 — 무료

브라우저에서 실제 코드를 작성하고 실행하며, 24/7 AI 튜터로부터 즉각적인 도움을 받고, 웹이나 앱에서 중단한 부분부터 계속 학습하세요.

코스: 30
레슨: 120

무료로 학습 시작 전체 코스 보기

자주 묻는 질문

“학습 반복: 손실, 옵티마이저 및 에포크” 강의는 무료인가요?

네 — “학습 반복: 손실, 옵티마이저 및 에포크” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Machine Learning Academy 강의 전체를 잠금 해제할 수 있습니다. Machine Learning Academy 강의에는 총 4개의 강의가 포함되어 있습니다.

“학습 반복: 손실, 옵티마이저 및 에포크”에서 뭘 배우나요?

학습자는 PyTorch 학습 반복을 작성합니다. zero_grad, 순전파, CrossEntropyLoss 계산, 역전파 및 optimizer.step을 수행하고 에포크별 손실과 정확도를 추적합니다. 브라우저에서 직접 실행하는 실습 코드로 Machine Learning Academy을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.

Machine Learning Academy을(를) 시작하는 데 경험이 필요한가요?

사전 경험은 필요하지 않습니다. CoddyKit의 Machine Learning Academy은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 4번째 강의입니다.

“학습 반복: 손실, 옵티마이저 및 에포크” 강의는 얼마나 걸리나요?

대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.

이 Machine Learning Academy 강의에서 코드를 작성하고 실행할 수 있나요?

네. 모든 Machine Learning Academy 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.

이 강의의 모든 강의

← Machine Learning Academy(으)로 돌아가기