트랜스포머 블록 내부

A Transformer is built by stacking the same block many times. Each block refines the representation a little more. 🧱

Every block has two parts: a multi-head attention sublayer followed by a small feed-forward network applied to each position.

The feed-forward sublayer is two linear layers with a nonlinearity between them, processing each token independently for extra power.

Each sublayer wraps its input in a residual connection: add the input back to the output so gradients flow and nothing is lost.

After adding the residual, layer normalization rescales the values. This keeps activations stable and speeds up training.

An encoder stack reads the input and builds rich context vectors. It uses self-attention so every token sees all the others.

A decoder generates output one token at a time. It adds masked attention plus cross-attention back to the encoder.

During generation, masking hides future tokens so the decoder cannot peek ahead and must predict the next word honestly.

Cross-attention lets the decoder query the encoder's outputs, connecting what it is writing to what it originally read.

Stacking many blocks lets early layers catch simple patterns and later layers build abstract meaning, much like deep vision networks.

BERT uses only the encoder, GPT uses only the decoder, and translation models use both. The block is the shared building unit. 🚀

Let us check the block structure.

You assembled the Transformer block: attention plus feed-forward, wrapped in residuals and layer norm, stacked into encoders and decoders. ✨