0Pricing
Machine Learning Academy · レッスン

CIFAR-10でCNNを構築して学習する

Conv2d-ReLU-MaxPoolブロックを積み重ね、特徴マップを平坦化して線形分類器を接続し、データ拡張を用いてCIFAR-10で学習します。

「CIFAR-10でCNNを構築して学習する」はCoddyKit上の無料Machine Learning Academyレッスンです。 これはレッスン3/4です。 下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。 これはMachine Learning Academy学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Machine Learning Academyコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

CIFAR-10: The Benchmark Dataset

CIFAR-10 is a classic image classification benchmark containing 60,000 colour images (32x32 pixels, RGB) in 10 classes: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. There are 50,000 training images and 10,000 test images. It is small enough to train on a laptop in a few hours but complex enough that simple models fail — making it ideal for learning CNN design. PyTorch makes it trivially available via torchvision.datasets.CIFAR10.

import torchvision
import torchvision.transforms as transforms

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        mean=(0.4914, 0.4822, 0.4465),
        std=(0.2023, 0.1994, 0.2010)
    )
])

train_set = torchvision.datasets.CIFAR10(
    root='./data', train=True,
    download=True, transform=transform
)
print('Training images:', len(train_set))   # 50000
print('Image shape:', train_set[0][0].shape) # (3, 32, 32)

Loading Data with DataLoader

After defining the dataset, wrap it in a DataLoader that handles batching, shuffling, and parallel loading. For CIFAR-10, a batch size of 64 or 128 is typical. Set shuffle=True for training data to randomise the order each epoch, and shuffle=False for the test set (order doesn't matter for evaluation). num_workers=2 loads data in parallel with training to reduce the GPU idle time during data fetching.

from torch.utils.data import DataLoader

train_loader = DataLoader(
    train_set,
    batch_size=128,
    shuffle=True,
    num_workers=2,
    pin_memory=True     # faster GPU transfer
)

# Peek at one batch
X_batch, y_batch = next(iter(train_loader))
print('Batch images:', X_batch.shape)   # (128, 3, 32, 32)
print('Batch labels:', y_batch.shape)   # (128,)

Designing the CNN Architecture

For CIFAR-10, a 3-block CNN works well: each block has two Conv-BN-ReLU layers followed by max pooling, doubling channels (32 -> 64 -> 128) while halving spatial size (32 -> 16 -> 8 -> 4). After the convolutional blocks, global average pooling collapses spatial dimensions and a linear layer maps to 10 class scores. This design has ~250K parameters — small enough to train quickly while achieving 80%+ accuracy without data augmentation.

import torch.nn as nn

class CIFAR10Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(),
            nn.Conv2d(32, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(),
            nn.MaxPool2d(2),           # 32->16
            nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(),
            nn.Conv2d(64, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(),
            nn.MaxPool2d(2),           # 16->8
            nn.Conv2d(64, 128, 3, padding=1), nn.BatchNorm2d(128), nn.ReLU(),
            nn.MaxPool2d(2),           # 8->4
        )
        self.classifier = nn.Sequential(
            nn.AdaptiveAvgPool2d((1, 1)),
            nn.Flatten(),
            nn.Linear(128, 10)
        )

    def forward(self, x):
        return self.classifier(self.features(x))

Setting Up Training Components

For CIFAR-10 classification, configure: CrossEntropyLoss as the criterion (raw logits for 10 classes), Adam or SGD with momentum as the optimizer, and a cosine annealing or step LR scheduler to reduce the learning rate during training. Move the model to GPU with .to(device). Print the total parameter count as a sanity check before starting the expensive training run.

import torch
import torch.nn as nn
import torch.optim as optim

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = CIFAR10Net().to(device)

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(
    model.parameters(),
    lr=0.1,
    momentum=0.9,
    weight_decay=5e-4
)
scheduler = optim.lr_scheduler.CosineAnnealingLR(
    optimizer, T_max=50
)

total_params = sum(p.numel() for p in model.parameters())
print(f'Parameters: {total_params:,}')   # ~250,000

The Training Epoch Function

Encapsulating one training epoch in a function makes the code reusable and clean. The function iterates over all batches, applies the 4-step update, accumulates loss and correct predictions, and returns the average loss and accuracy. Moving both X_batch and y_batch to the device inside the loop is the correct pattern — prefetching with pin_memory=True in the DataLoader speeds up this transfer.

import torch

def train_epoch(model, loader, criterion, optimizer, device):
    model.train()
    total_loss, correct, total = 0.0, 0, 0
    for X, y in loader:
        X, y = X.to(device), y.to(device)
        optimizer.zero_grad()
        logits = model(X)
        loss = criterion(logits, y)
        loss.backward()
        optimizer.step()
        total_loss += loss.item() * X.size(0)
        correct += (logits.argmax(1) == y).sum().item()
        total += X.size(0)
    return total_loss / total, correct / total

The Validation Epoch Function

The validation function is identical to the training function except: model.eval() switches off Dropout and BatchNorm stochasticity; torch.no_grad() disables gradient tracking for speed; and there is no optimizer step. The returned validation accuracy and loss are used to track generalisation performance, detect overfitting, and drive learning rate scheduling via ReduceLROnPlateau.

import torch

def eval_epoch(model, loader, criterion, device):
    model.eval()
    total_loss, correct, total = 0.0, 0, 0
    with torch.no_grad():
        for X, y in loader:
            X, y = X.to(device), y.to(device)
            logits = model(X)
            loss = criterion(logits, y)
            total_loss += loss.item() * X.size(0)
            correct += (logits.argmax(1) == y).sum().item()
            total += X.size(0)
    return total_loss / total, correct / total

The Full Training Loop

With training and validation epoch functions defined, the outer loop runs for num_epochs iterations. After each epoch, step the LR scheduler and save the model if validation accuracy improves. Printing metrics every epoch provides visibility — you should see training accuracy climb from ~30% in the first epoch to 90%+ by epoch 50 for a properly tuned CNN. Validation accuracy typically lags training by 5-10 percentage points.

num_epochs = 50
best_val_acc = 0.0

for epoch in range(num_epochs):
    tr_loss, tr_acc = train_epoch(
        model, train_loader, criterion, optimizer, device)
    val_loss, val_acc = eval_epoch(
        model, val_loader, criterion, device)
    scheduler.step()

    print(f'Epoch {epoch+1:03d}: '
          f'tr_loss={tr_loss:.3f} tr_acc={tr_acc:.3f} '
          f'val_loss={val_loss:.3f} val_acc={val_acc:.3f}')

    if val_acc > best_val_acc:
        best_val_acc = val_acc
        torch.save(model.state_dict(), 'best_cifar10.pt')

Data Augmentation to Improve Accuracy

Data augmentation applies random transformations to training images, artificially expanding the dataset and making the model more robust. For CIFAR-10, random horizontal flip, random crop (with padding), and color jitter are standard. Augmentation improves accuracy by 3-5 percentage points typically. Apply augmentation only to the training transform — the test transform uses only normalisation for deterministic evaluation.

import torchvision.transforms as transforms

train_transform = transforms.Compose([
    transforms.RandomCrop(32, padding=4),       # shift by 4px
    transforms.RandomHorizontalFlip(),           # mirror 50%
    transforms.ColorJitter(
        brightness=0.2, contrast=0.2,
        saturation=0.2, hue=0.1
    ),
    transforms.ToTensor(),
    transforms.Normalize(
        (0.4914, 0.4822, 0.4465),
        (0.2023, 0.1994, 0.2010)
    )
])

test_transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(
        (0.4914, 0.4822, 0.4465),
        (0.2023, 0.1994, 0.2010)
    )
])

Interpreting Training Curves

Learning to read training curves is essential for CNN development. A converging model shows both training and validation loss decreasing together. Overfitting shows training loss near zero while validation loss plateaus or rises. Underfitting shows both losses high and flat. For CIFAR-10, a healthy 50-epoch run should show: epoch 1 ~35% accuracy, epoch 10 ~70%, epoch 50 ~85-90% validation accuracy with the described architecture and augmentation.

# Typical CIFAR-10 accuracy milestones
milestones = {
    'Epoch 1':  'val_acc ~35%  (random = 10%)',
    'Epoch 5':  'val_acc ~60%  (basic patterns learned)',
    'Epoch 10': 'val_acc ~70%  (edges, textures)',
    'Epoch 20': 'val_acc ~78%  (object parts)',
    'Epoch 50': 'val_acc ~85%  (with augmentation)',
    'With ResNet18': 'val_acc ~93%',
    'State of art': 'val_acc ~99%  (huge ensembles)'
}
for epoch, note in milestones.items():
    print(f'{epoch}: {note}')

Per-Class Accuracy Analysis

Overall accuracy hides per-class performance differences. Some CIFAR-10 classes are harder to distinguish — 'cat' vs 'dog' and 'automobile' vs 'truck' are common confusion pairs. Computing a per-class accuracy or plotting the confusion matrix reveals which classes the model struggles with, guiding targeted data collection or augmentation strategies. Use sklearn.metrics.confusion_matrix on collected predictions and labels.

import torch

classes = ['airplane', 'auto', 'bird', 'cat', 'deer',
           'dog', 'frog', 'horse', 'ship', 'truck']

# Collect all predictions
all_preds, all_labels = [], []
model.eval()
with torch.no_grad():
    for X, y in test_loader:
        X = X.to(device)
        preds = model(X).argmax(1).cpu()
        all_preds.extend(preds.tolist())
        all_labels.extend(y.tolist())

# Per-class accuracy
for i, cls in enumerate(classes):
    mask = [l == i for l in all_labels]
    correct = sum(p == l for p, l in zip(all_preds, all_labels) if l == i)
    total = sum(mask)
    print(f'{cls}: {correct}/{total} = {correct/total:.1%}')

CutMix and Mixup Augmentations

Beyond standard augmentation, advanced techniques improve CNN performance further. Mixup linearly interpolates between two training images and their labels: the model must predict a blend of both classes. CutMix pastes a rectangular patch from one image onto another, assigning labels proportionally to the area. Both act as strong regularisers and reduce overfitting. They are standard in state-of-the-art CIFAR-10 training runs, adding 1-3% accuracy over random crop + flip alone.

import torch

def mixup_batch(X, y, alpha=0.2, num_classes=10):
    lam = torch.distributions.Beta(alpha, alpha).sample()
    idx = torch.randperm(X.size(0))
    X_mix = lam * X + (1 - lam) * X[idx]
    # Soft labels: blend of one-hot vectors
    y_onehot = torch.zeros(X.size(0), num_classes)
    y_onehot.scatter_(1, y.unsqueeze(1), 1)
    y_onehot2 = y_onehot[idx]
    y_mix = lam * y_onehot + (1 - lam) * y_onehot2
    return X_mix, y_mix

# Use with soft-label cross entropy
# loss = -(y_mix * F.log_softmax(logits, dim=1)).sum(dim=1).mean()

Quick Check

Test your understanding of Machine Learning with Python concepts from this lesson.

Lesson Recap

In this lesson you learned: CIFAR-10 is the standard 10-class image benchmark with 50K train and 10K test 32x32 RGB images, a 3-block CNN with BatchNorm achieves ~85% accuracy after 50 epochs with standard augmentation, and data augmentation (random crop, horizontal flip, colour jitter) is essential for closing the gap between training and test accuracy. Next up we explore data augmentation transforms in detail for building more robust models.

よくある質問

「CIFAR-10でCNNを構築して学習する」レッスンは無料ですか？

はい。「CIFAR-10でCNNを構築して学習する」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Machine Learning Academyコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Machine Learning Academyコースには全4レッスンが含まれています。

「CIFAR-10でCNNを構築して学習する」で何を学びますか？

Conv2d-ReLU-MaxPoolブロックを積み重ね、特徴マップを平坦化して線形分類器を接続し、データ拡張を用いてCIFAR-10で学習します。 ブラウザで直接実行するハンズオンコードでMachine Learning Academyを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Machine Learning Academyを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのMachine Learning Academyは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。 これはレッスン3/4です。

「CIFAR-10でCNNを構築して学習する」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このMachine Learning Academyレッスンでコードを書いて実行できますか？

はい。すべてのMachine Learning Academyレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

  1. 畳み込みとフィルター：エッジとパターンの検出
  2. プーリング層：空間的ダウンサンプリングと不変性
  3. CIFAR-10でCNNを構築して学習する
  4. データ拡張：頑健性を高める変換
← Machine Learning Academyに戻る