Elixir & Phoenix: Scalable Backend Development · 课时

GenStage 与背压流水线

使用 GenStage 构建由需求驱动的数据流水线，让消费者通过背压控制流量，避免分布式系统过载。

第 4 / 4 课13 个步骤

GenStage 与背压流水线 是 CoddyKit 上的免费 Elixir & Phoenix: Scalable Backend Development 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Elixir & Phoenix: Scalable Backend Development 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Elixir & Phoenix: Scalable Backend Development 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

The Overload Problem

When a fast producer feeds a slow consumer, messages pile up and memory explodes. Backpressure solves this: consumers signal how much work they can handle, and producers send only that much.

What is GenStage?

GenStage is an Elixir behaviour for building staged, demand-driven pipelines. Data flows from producers to consumers, with optional producer-consumer stages in between.

The Three Stage Types

Every GenStage process plays one role:

Producer: emits events on demand
ProducerConsumer: receives, transforms, and re-emits
Consumer: receives and processes events

Defining a Producer

A producer implements init/1 and handle_demand/2. It returns events only when consumers ask for them.

defmodule Counter do
  use GenStage
  def init(start), do: {:producer, start}
  def handle_demand(demand, state) do
    events = Enum.to_list(state..(state + demand - 1))
    {:noreply, events, state + demand}
  end
end

Defining a Consumer

A consumer implements handle_events/3. It returns no events itself, only acknowledging the demand it processed.

defmodule Printer do
  use GenStage
  def init(_), do: {:consumer, :ok}
  def handle_events(events, _from, state) do
    for e <- events, do: IO.inspect(e)
    {:noreply, [], state}
  end
end

Subscribing Stages

Connect a consumer to a producer with GenStage.sync_subscribe/2. The max_demand option caps how many events flow at once.

{:ok, prod} = GenStage.start_link(Counter, 0)
{:ok, cons} = GenStage.start_link(Printer, :ok)
GenStage.sync_subscribe(cons, to: prod, max_demand: 10)

How Backpressure Works

The consumer requests up to max_demand events. The producer can never send more than was requested, so a slow consumer naturally throttles a fast producer. No buffering blowup.

Producer-Consumer Stages

A middle stage transforms data. It implements handle_events/3 but returns transformed events for the next stage.

def handle_events(events, _from, state) do
  doubled = Enum.map(events, &(&1 * 2))
  {:noreply, doubled, state}
end

ConsumerSupervisor

For concurrent processing, ConsumerSupervisor spawns a short-lived child process per event, bounded by demand. This parallelizes work while keeping backpressure intact.

Flow for Parallel Pipelines

The Flow library builds on GenStage to offer map/reduce-style parallel data processing with partitioning — ideal for crunching large collections across cores.

File.stream!("big.csv")
|> Flow.from_enumerable()
|> Flow.map(&parse_line/1)
|> Flow.partition()
|> Enum.to_list()

When to Reach for GenStage

Use GenStage when you have:

A rate mismatch between data source and processing
Streaming data that must not overwhelm memory
Multi-stage transformation pipelines

Quick Check

Test your GenStage knowledge.

在交互式课程中回答

Recap

You learned demand-driven pipelines:

GenStage has producers, producer-consumers, and consumers
Consumers request bounded demand for backpressure
Subscribe stages with sync_subscribe and max_demand
ConsumerSupervisor parallelizes per-event work
Flow builds parallel map/reduce pipelines on top

Backpressure keeps distributed systems stable under load.

免费开始

用 AI 导师学习 Elixir — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程: 12
课程: 48

免费开始学习查看完整课程

常见问题解答

「GenStage 与背压流水线」课时是免费的吗？

是的 — 「GenStage 与背压流水线」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Elixir & Phoenix: Scalable Backend Development 课程的其余内容，请升级到 CoddyKit PRO。 Elixir & Phoenix: Scalable Backend Development 课程共包含 4 节课。

「GenStage 与背压流水线」这节课中我会学到什么？

使用 GenStage 构建由需求驱动的数据流水线，让消费者通过背压控制流量，避免分布式系统过载。你通过在浏览器中直接运行的动手代码来练习 Elixir & Phoenix: Scalable Backend Development，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Elixir & Phoenix: Scalable Backend Development 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Elixir & Phoenix: Scalable Backend Development 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「GenStage 与背压流水线」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Elixir & Phoenix: Scalable Backend Development 课中编写并运行代码吗？

能。每节 Elixir & Phoenix: Scalable Backend Development 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Elixir & Phoenix: Scalable Backend Development