Inside the Transformer Block
How the full architecture fits together.
Inside the Transformer Block is a free NLP Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Stacking Blocks
A Transformer is built by stacking the same block many times. Each block refines the representation a little more. 🧱
Two Main Sublayers
Every block has two parts: a multi-head attention sublayer followed by a small feed-forward network applied to each position.
The Feed-Forward Network
The feed-forward sublayer is two linear layers with a nonlinearity between them, processing each token independently for extra power.
h = relu(x @ W1 + b1) @ W2 + b2Residual Connections
Each sublayer wraps its input in a residual connection: add the input back to the output so gradients flow and nothing is lost.
out = x + sublayer(x)Layer Normalization
After adding the residual, layer normalization rescales the values. This keeps activations stable and speeds up training.
out = layer_norm(x + sublayer(x))Encoder Blocks
An encoder stack reads the input and builds rich context vectors. It uses self-attention so every token sees all the others.
Decoder Blocks
A decoder generates output one token at a time. It adds masked attention plus cross-attention back to the encoder.
Masked Attention
During generation, masking hides future tokens so the decoder cannot peek ahead and must predict the next word honestly.
Cross-Attention
Cross-attention lets the decoder query the encoder's outputs, connecting what it is writing to what it originally read.
Depth Brings Power
Stacking many blocks lets early layers catch simple patterns and later layers build abstract meaning, much like deep vision networks.
Encoder, Decoder, or Both
BERT uses only the encoder, GPT uses only the decoder, and translation models use both. The block is the shared building unit. 🚀
Quick Check
Let us check the block structure.
Recap
You assembled the Transformer block: attention plus feed-forward, wrapped in residuals and layer norm, stacked into encoders and decoders. ✨
Frequently asked questions
Is the “Inside the Transformer Block” lesson free?
Yes — the full text of “Inside the Transformer Block” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Inside the Transformer Block”?
How the full architecture fits together. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Inside the Transformer Block” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- The Idea of Attention
- Self-Attention, Step by Step
- Multi-Head Attention and Positions
- Inside the Transformer Block