collate_fn สำหรับอินพุตความยาวต่างกัน

Stacking into a batch needs every sample the same shape. But sentences and audio clips have different lengths, so the default collate step fails. 🧩

The DataLoader gathers a list of samples and passes them to collate_fn, which merges them into one batch. By default it simply stacks tensors.

Try to stack a length-5 and a length-8 sequence and PyTorch raises a shape error. Ragged lengths are exactly the case a custom collate must handle.

You pass a function to the DataLoader's collate_fn argument. It receives a list of samples and returns whatever batch shape your model expects.

Inside your function, unzip the list of pairs into separate sequences and labels. Now you can treat each group on its own before merging.

The trick for variable lengths is padding: extend every sequence to the longest one with a filler value, so they finally share a shape.

PyTorch ships pad_sequence, which pads a list of tensors to equal length and stacks them. Set batch_first so the batch dimension comes first.

Padding adds fake tokens, so also return each sequence's true length. Your model uses these to ignore the padded positions during the forward pass.

Labels are usually fixed size, so a normal stack works for them. Return the padded inputs, the lengths, and the stacked labels together.

Later you build a mask from the lengths so the loss and attention skip padded slots. Padding fills shape without polluting the gradients.

With a custom collate_fn, the same DataLoader handles text, audio, and graphs. You control exactly how loose samples become one tidy batch.

Why do variable-length sequences need a custom collate_fn?

A custom collate_fn turns a list of uneven samples into one batch, usually by padding sequences to equal length and tracking their real sizes. 🎉