멀티헤드 어텐션과 위치 정보

A single attention head can track only one kind of relationship at a time. Real language needs many patterns noticed at once. 🧠

Multi-head attention runs several attention computations in parallel, each with its own learned projections and its own focus.

One head might track subject-verb links while another follows pronouns. Together they capture a far richer picture of the sentence.

The model splits its vector size across heads, so each head works in a smaller subspace. The total compute stays roughly the same.

After each head produces an output, you concatenate them and pass the result through one more linear layer to mix the views.

Self-attention treats input as a set, so by itself it cannot tell "dog bites man" from "man bites dog." It is order-blind.

To fix this, we inject a positional encoding into each word so the model knows where every token sits in the sequence.

The original Transformer uses fixed sine and cosine waves of different frequencies to give each position a unique, smooth signature.

Position vectors are added to the word embeddings, not stuck on the end. So each token carries both meaning and place together.

Many modern models replace fixed waves with learned position embeddings, trained alongside everything else for flexibility.

Multi-head attention sees many relationships; positional encodings restore order. Together they let the Transformer truly understand sequences. ✨

Let us test positions and heads.

You saw how multi-head attention captures many relationships in parallel, while positional encodings give the model a sense of order. 🎯