collate_fn for Variable-Length Inputs
Pad and stack ragged samples.
collate_fn for Variable-Length Inputs is a free Deep Learning Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Deep Learning Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
When Samples Don't Match
Stacking into a batch needs every sample the same shape. But sentences and audio clips have different lengths, so the default collate step fails. 🧩
What collate_fn Does
The DataLoader gathers a list of samples and passes them to collate_fn, which merges them into one batch. By default it simply stacks tensors.
Ragged Inputs Break Stacking
Try to stack a length-5 and a length-8 sequence and PyTorch raises a shape error. Ragged lengths are exactly the case a custom collate must handle.
Write Your Own collate_fn
You pass a function to the DataLoader's collate_fn argument. It receives a list of samples and returns whatever batch shape your model expects.
loader = DataLoader(ds, batch_size=4, collate_fn=my_collate)Step One: Split the List
Inside your function, unzip the list of pairs into separate sequences and labels. Now you can treat each group on its own before merging.
def my_collate(batch):
seqs, labels = zip(*batch)Pad to the Longest
The trick for variable lengths is padding: extend every sequence to the longest one with a filler value, so they finally share a shape.
pad_sequence Does It for You
PyTorch ships pad_sequence, which pads a list of tensors to equal length and stacks them. Set batch_first so the batch dimension comes first.
from torch.nn.utils.rnn import pad_sequence
padded = pad_sequence(seqs, batch_first=True)Remember the Real Lengths
Padding adds fake tokens, so also return each sequence's true length. Your model uses these to ignore the padded positions during the forward pass.
lengths = torch.tensor([len(s) for s in seqs])Stack the Labels
Labels are usually fixed size, so a normal stack works for them. Return the padded inputs, the lengths, and the stacked labels together.
labels = torch.stack(labels)
return padded, lengths, labelsMask Out the Padding
Later you build a mask from the lengths so the loss and attention skip padded slots. Padding fills shape without polluting the gradients.
One Function, Any Shape
With a custom collate_fn, the same DataLoader handles text, audio, and graphs. You control exactly how loose samples become one tidy batch.
Quick Check
Why do variable-length sequences need a custom collate_fn?
Recap
A custom collate_fn turns a list of uneven samples into one batch, usually by padding sequences to equal length and tracking their real sizes. 🎉
Frequently asked questions
Is the “collate_fn for Variable-Length Inputs” lesson free?
Yes — the full text of “collate_fn for Variable-Length Inputs” is free to read here on the web, and the Deep Learning Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Deep Learning Academy course, upgrade to CoddyKit PRO.
What will I learn in “collate_fn for Variable-Length Inputs”?
Pad and stack ragged samples. You practise Deep Learning Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Deep Learning Academy?
No prior experience is required. Deep Learning Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “collate_fn for Variable-Length Inputs” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Deep Learning Academy lesson?
Yes. Every Deep Learning Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Write a Custom Dataset Class
- Batching, Shuffling & num_workers
- collate_fn for Variable-Length Inputs
- Normalize and Standardize Inputs