Deep Learning Academy · 课时

编写自定义数据集类

实现 __len__ 与 __getitem__

编写自定义数据集类 是 CoddyKit 上的免费 Deep Learning Academy 课时。这是第 1 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Deep Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Deep Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Your Data Needs a Front Door

Before a model can learn, PyTorch needs a clean way to reach your samples one at a time. That front door is a Dataset class. 🚪

Start by Subclassing

You build a custom dataset by subclassing torch.utils.data.Dataset. PyTorch then knows exactly how to ask your object for data.

from torch.utils.data import Dataset

class MyData(Dataset):
    pass

Stash Your Data in init

The __init__ method runs once when you create the dataset. Use it to load file paths, arrays, or labels into the object's fields.

def __init__(self, X, y):
    self.X = X
    self.y = y

Two Methods Make It Work

A working dataset only needs two methods: __len__ to report its size and __getitem__ to fetch one sample. That is the whole contract.

len Counts Your Samples

The __len__ method returns how many samples you have. PyTorch reads this to know when an epoch ends and how far an index can go.

def __len__(self):
    return len(self.X)

getitem Returns One Sample

Given an index, __getitem__ returns a single sample, usually a feature and its label. This is where one row of data is handed over.

def __getitem__(self, idx):
    return self.X[idx], self.y[idx]

Return Tensors, Not Lists

__getitem__ should hand back tensors so the model can use them directly. Convert NumPy arrays or Python lists right here if needed.

import torch
x = torch.tensor(self.X[idx], dtype=torch.float32)

Lazy Loading for Big Data

For huge datasets, do not load everything in __init__. Instead read each file inside __getitem__ so only one sample sits in memory at a time.

Apply Transforms Per Sample

__getitem__ is the natural place to apply a transform, like resizing an image. Store the transform in __init__, then call it before returning.

if self.transform:
    x = self.transform(x)

Index It Like a List

Once built, your dataset behaves like a list. Calling len(ds) or ds[0] just triggers the two methods you defined. Test it before training.

ds = MyData(X, y)
print(len(ds), ds[0])

Now It Plugs Into Everything

This tidy interface is why a custom Dataset drops straight into a DataLoader. You write two methods and the rest of PyTorch just works.

Quick Check

Which method does PyTorch call to fetch a single sample by index?

Recap

A custom dataset subclasses Dataset and defines two methods: __len__ for its size and __getitem__ to return one sample. Two methods, full power. 🎉

常见问题解答