시퀀스 패킹과 패딩 처리

To process many sequences at once, a batch must be a neat rectangle. But real sentences have different lengths, so they don't line up.

The fix is padding: add filler tokens, usually zeros, to short sequences so every row in the batch is the same length.

PyTorch's pad_sequence stacks a list of variable-length tensors into one padded batch in a single call.

Those filler tokens are fake. If the RNN processes them anyway, it burns compute and can let the padding pollute the hidden state.

Before padding, record each sequence's true length. You'll hand these lengths to PyTorch so it knows where the real data ends.

pack_padded_sequence turns the padded batch plus lengths into a compact form the RNN can run without touching the padding.

Feed the packed object straight into nn.LSTM or nn.GRU. The cell skips padded steps, so memory stays clean and clear.

To read per-step outputs again, call pad_packed_sequence to restore the rectangular padded shape after the RNN finishes.

Classic packing wants sequences sorted longest first. Set enforce_sorted to False and PyTorch handles unsorted batches for you.

Even with packing, ignore padded positions when scoring. A mask or ignore_index keeps fake tokens out of the loss.

Do the padding inside a custom collate_fn so every batch arrives already padded, with lengths ready for packing.

Pad to align a batch, track true lengths, then pack so the RNN ignores filler. Mask padded tokens in the loss too. ✅