0Pricing
Deep Learning Academy · 课时

堆叠 Transformer 编码器模块

组合注意力、前馈网络与归一化

堆叠 Transformer 编码器模块 是 CoddyKit 上的免费 Deep Learning Academy 课时。 这是第 4 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Deep Learning Academy 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Deep Learning Academy 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

The Building Block

A transformer is just one encoder block repeated. Learn the block and you understand the whole tower, from tiny models to giant ones.

Two Sub-Layers

Each block has two parts: a multi-head attention sub-layer, then a small feedforward network. Both wrapped with residuals and normalization.

Attention First

The block starts with self-attention, letting every token mix in context from the rest of the sequence before any further processing.

attn_out, _ = self.attn(x, x, x)

The Residual Connection

A residual adds the sub-layer's input back to its output. This shortcut lets gradients flow and keeps deep stacks trainable.

x = x + attn_out

Layer Normalization

After adding the residual, layer norm rescales each token's features to a stable distribution, steadying training across layers.

x = self.norm1(x)

The Feedforward Net

Next comes a position-wise feedforward network: expand to a wider hidden size, apply a nonlinearity, then project back down.

ff = nn.Sequential(nn.Linear(d, 4*d), nn.GELU(), nn.Linear(4*d, d))

Per-Token Processing

The feedforward layer treats each token independently. Attention shared information; this step refines each token on its own.

Second Residual and Norm

The feedforward output gets the same treatment: a residual add plus another layer norm, finishing the block.

x = self.norm2(x + ff(x))

Stack Them Deep

Stack many identical blocks and the model builds richer representations layer by layer. Depth is where transformer power comes from.

layers = nn.ModuleList([Block(d) for _ in range(N)])

Pre-Norm vs Post-Norm

Many modern models apply layer norm before each sub-layer instead of after. Pre-norm trains more stably in very deep stacks.

Use the Built-in

PyTorch gives you nn.TransformerEncoderLayer and nn.TransformerEncoder, so you can assemble a full stack in just a couple of lines.

layer = nn.TransformerEncoderLayer(d_model, nhead)
enc = nn.TransformerEncoder(layer, num_layers=6)

Quick Check

Let's review the parts of an encoder block.

Recap

You assembled an encoder block: attention, residual, norm, feedforward, residual, norm. Stack it deep and you have a transformer. Amazing work!

常见问题解答

「堆叠 Transformer 编码器模块」课时是免费的吗？

是的 — 「堆叠 Transformer 编码器模块」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Deep Learning Academy 课程的其余内容，请升级到 CoddyKit PRO。 Deep Learning Academy 课程共包含 4 节课。

「堆叠 Transformer 编码器模块」这节课中我会学到什么？

组合注意力、前馈网络与归一化 你通过在浏览器中直接运行的动手代码来练习 Deep Learning Academy，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Deep Learning Academy 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Deep Learning Academy 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 4 节课，共 4 节。

「堆叠 Transformer 编码器模块」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Deep Learning Academy 课中编写并运行代码吗？

能。每节 Deep Learning Academy 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

  1. 自注意力：查询、键与值
  2. 缩放点积与多头注意力
  3. 用于表达顺序的位置编码
  4. 堆叠 Transformer 编码器模块
← 返回 Deep Learning Academy