0Pricing
Next.js 15 Fullstack (App Router + Server Actions) · 课时

逐令牌流式传输人工智能响应

使用具备背压感知能力的读取器和中止处理，将 LLM 补全内容流式传输到界面。

逐令牌流式传输人工智能响应 是 CoddyKit 上的免费 Next.js 15 Fullstack (App Router + Server Actions) 课时。 这是第 3 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Next.js 15 Fullstack (App Router + Server Actions) 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Next.js 15 Fullstack (App Router + Server Actions) 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Stream AI Responses?

When you call a large language model (LLM) like GPT-4 or Claude, the model generates tokens one by one. A typical response may take 5–20 seconds to complete. If you wait for the full response before sending anything to the client, the user stares at a blank screen the entire time.

Streaming solves this: you pipe each token to the browser as it is produced, creating the familiar "typewriter" effect used by ChatGPT, Claude.ai, and Gemini.

  • Perceived latency drops from time-to-full-response to time-to-first-token (often under 300 ms).
  • Users can start reading and even abort early if the answer is already clear.
  • Server memory stays flat — you never buffer the whole response.

In Next.js 15 the primitives you need are ReadableStream, the Web Streams API, and StreamingTextResponse (or a plain Response with a stream body).

How LLM SDKs Expose Streams

Most LLM SDKs return an async iterable or a ReadableStream when you pass stream: true. The Vercel AI SDK unifies these under a single interface.

With the official OpenAI SDK you receive a stream of ChatCompletionChunk objects. Each chunk carries a delta.content string that may be one token, a few characters, or an empty string at the end.

  • OpenAI SDK: openai.chat.completions.create({ stream: true }) returns an AsyncIterable.
  • Vercel AI SDK: streamText() returns a result with result.toDataStreamResponse() ready for Next.js Route Handlers.
  • Anthropic SDK: client.messages.stream() returns an async iterable of MessageStreamEvent.

Regardless of the SDK, the pattern is the same: iterate over chunks, encode each piece, and enqueue it into a ReadableStream that becomes the HTTP response body.

Creating a Streaming Route Handler

A Next.js 15 Route Handler returns a standard Response. You can pass a ReadableStream as the body to stream data to the client. The ReadableStream constructor accepts a start(controller) callback where you push chunks with controller.enqueue() and signal completion with controller.close().

Below is a minimal Route Handler at app/api/chat/route.ts that streams an OpenAI completion token by token.

// app/api/chat/route.ts
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    stream: true,
    messages: [{ role: 'user', content: prompt }],
  });

  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          controller.enqueue(encoder.encode(text));
        }
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Transfer-Encoding': 'chunked',
    },
  });
}

Reading the Stream on the Client

On the browser side you use the Fetch API together with a ReadableStreamDefaultReader to consume the stream incrementally. The key steps are:

  1. Call fetch() — do not await response.json() (that buffers everything).
  2. Get the reader: response.body!.getReader().
  3. Loop with reader.read() until done === true.
  4. Decode each Uint8Array chunk with TextDecoder and append to state.

This is called a pull-based read loop. You pull the next chunk only after you have processed the previous one, giving you natural backpressure.

// Standalone client utility — no React dependency
async function streamCompletion(
  prompt: string,
  onChunk: (text: string) => void
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    const text = decoder.decode(value, { stream: true });
    onChunk(text);
  }
}

// Usage example (TypeScript, no browser DOM required for the logic):
// streamCompletion('Hello!', (chunk) => process.stdout.write(chunk));

Abort Handling with AbortController

Users frequently stop a generation mid-stream. Without abort handling, the server keeps calling the LLM and the client leaks a reader that never closes.

The fix is AbortController. You pass its signal to fetch(). When you call controller.abort(), the fetch throws an AbortError and the browser cancels the underlying TCP read, which in turn triggers backpressure cancellation upstream.

  • Always wrap the read loop in a try/finally block so reader.releaseLock() is always called.
  • On the server, pass the request's signal to the OpenAI SDK so the LLM call itself is cancelled — this saves tokens and money.
// Client-side abort-aware stream reader
async function streamWithAbort(
  prompt: string,
  onChunk: (text: string) => void,
  signal: AbortSignal
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
    signal, // <-- pass AbortSignal to fetch
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(decoder.decode(value, { stream: true }));
    }
  } finally {
    reader.releaseLock(); // always release even on abort
  }
}

Propagating Abort to the LLM on the Server

When the client aborts, the fetch connection closes. Next.js 15 Route Handlers expose req.signal — a native AbortSignal that fires when the client disconnects. Pass this signal to the LLM SDK to cancel the upstream API call immediately.

Inside the ReadableStream constructor you can also implement a cancel() method that cleans up any ongoing work when the stream is cancelled by the consumer.

// app/api/chat/route.ts — with server-side abort propagation
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create(
    {
      model: 'gpt-4o-mini',
      stream: true,
      messages: [{ role: 'user', content: prompt }],
    },
    { signal: req.signal } // propagate client disconnect signal
  );

  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of completion) {
          if (req.signal.aborted) break;
          const text = chunk.choices[0]?.delta?.content ?? '';
          if (text) controller.enqueue(encoder.encode(text));
        }
      } catch (err) {
        if ((err as Error).name !== 'AbortError') throw err;
      } finally {
        controller.close();
      }
    },
    cancel() {
      // Called when the consumer (browser) cancels the stream
      completion.controller.abort();
    },
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

Using the Vercel AI SDK for Simpler Streaming

The Vercel AI SDK (ai package) removes most of the boilerplate. streamText() handles the stream construction, abort propagation, and error boundaries for you. It also adds a structured data stream format that the client-side useChat hook can parse.

The result object exposes result.toDataStreamResponse() which returns a fully-formed Response ready to return from your Route Handler.

// app/api/chat/route.ts — Vercel AI SDK
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request): Promise<Response> {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
    abortSignal: req.signal, // automatic abort propagation
  });

  // toDataStreamResponse() streams tokens in Vercel AI data-stream format
  return result.toDataStreamResponse();
}

The useChat Hook on the Client

When the server uses the Vercel AI SDK data-stream format, the companion useChat hook on the client handles everything: fetching, reading the stream, appending tokens, and exposing an isLoading flag plus a stop() function for abort.

This is the recommended pattern for chat UIs in Next.js 15 App Router projects. The hook is framework-agnostic enough to work in both Client Components and Server Components that hydrate client islands.

// app/chat/page.tsx — Client Component using useChat
'use client';

import { useChat } from 'ai/react';

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, stop } =
    useChat({ api: '/api/chat' });

  return (
    <main style={{ maxWidth: 600, margin: '0 auto', padding: 24 }}>
      <ul>
        {messages.map((m) => (
          <li key={m.id}>
            <strong>{m.role}:</strong> {m.content}
          </li>
        ))}
      </ul>
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Ask anything..." />
        <button type="submit" disabled={isLoading}>Send</button>
        {isLoading && (
          <button type="button" onClick={stop}>Stop</button>
        )}
      </form>
    </main>
  );
}

Backpressure: Why It Matters

Backpressure is the mechanism that prevents a fast producer (the LLM API) from overwhelming a slow consumer (the browser rendering pipeline or a slow network).

In the Web Streams API, backpressure is built in:

  • Each call to reader.read() waits for the consumer to be ready before requesting the next chunk from the underlying source.
  • The ReadableStream internal queue holds a limited number of chunks (controlled by QueuingStrategy). When the queue is full, the producer is paused automatically.
  • If you use for await...of on a stream, the JavaScript runtime handles pull-pacing for you — you get one chunk per iteration, naturally throttled.

You rarely need to configure a custom QueuingStrategy for AI token streams because LLM responses are slow enough that the default high-watermark (1 chunk) is sufficient.

// Demonstrating a custom ByteLengthQueuingStrategy (illustrative)
const strategy = new ByteLengthQueuingStrategy({ highWaterMark: 1024 }); // 1 KB buffer

const stream = new ReadableStream(
  {
    start(controller) {
      // producer: enqueue only when consumer is ready
      controller.enqueue(new TextEncoder().encode('Hello '));
      controller.enqueue(new TextEncoder().encode('world!'));
      controller.close();
    },
  },
  strategy
);

// Consumer: pull-based, respects backpressure
const reader = stream.getReader();
async function drain() {
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    console.log(new TextDecoder().decode(value));
  }
}
drain();

Server-Sent Events vs Raw Streaming

There are two common wire formats for streaming AI responses to the browser:

  • Raw byte stream (Content-Type: text/plain): the simplest approach. Each chunk is raw UTF-8 text. Used in the earlier examples. Works well for simple prose but carries no metadata (token counts, finish reasons, etc.).
  • Server-Sent Events (SSE) (Content-Type: text/event-stream): a standardised line-based format. Each event is data: <payload>\n\n. Browsers have a built-in EventSource API, but it only supports GET. For POST-based chat you use a custom SSE parser on top of fetch.

The Vercel AI SDK data-stream format is an SSE-like protocol that encodes token deltas, tool calls, and metadata as structured lines. Use raw streaming for simple use-cases; use the AI SDK protocol when you need structured events.

// app/api/chat-sse/route.ts — manual SSE format over POST
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };
  const encoder = new TextEncoder();

  const sseStream = new ReadableStream({
    async start(controller) {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4o-mini',
        stream: true,
        messages: [{ role: 'user', content: prompt }],
      });

      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          // SSE line: data: <payload>\n\n
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ text })}\n\n`));
        }
      }
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });

  return new Response(sseStream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

Error Handling and Finish Reasons

Streaming introduces new error scenarios that do not exist with a single-shot response:

  • Mid-stream network drop: the read loop throws; catch it and show a "Connection lost" UI.
  • LLM rate-limit (429): the HTTP response itself is non-2xx; check response.ok before reading the body.
  • Content filter stop: the stream ends normally but the last chunk's finish_reason is 'content_filter' — you must read finish reasons out-of-band or from a final SSE event.
  • Token limit reached: finish_reason === 'length' — inform the user the response was truncated.

Always implement a try/catch around the read loop and display graceful fallback UI. Never leave the stream reader locked on error.

// Robust client-side stream reader with error handling
async function safeStream(
  prompt: string,
  onChunk: (t: string) => void,
  onError: (msg: string) => void,
  signal: AbortSignal
): Promise<void> {
  let reader: ReadableStreamDefaultReader<Uint8Array> | null = null;
  try {
    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
      signal,
    });

    if (!res.ok) {
      onError(`Server error: ${res.status} ${res.statusText}`);
      return;
    }

    reader = res.body!.getReader();
    const dec = new TextDecoder();

    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(dec.decode(value, { stream: true }));
    }
  } catch (err) {
    if ((err as Error).name === 'AbortError') return; // user-initiated, silent
    onError((err as Error).message);
  } finally {
    reader?.releaseLock();
  }
}

Knowledge Check: Abort Handling

Consider a Next.js 15 Route Handler that streams an OpenAI completion. A user clicks Stop in the UI, which calls controller.abort() on an AbortController whose signal was passed to fetch(). Which additional step is most critical to prevent wasted LLM tokens and unnecessary server compute?

Lesson Recap

In this lesson you learned how to stream LLM completions token-by-token in a Next.js 15 App Router application:

  • Why stream: reduces perceived latency from seconds to milliseconds by sending the first token before the full response is ready.
  • Route Handler: return a new Response(readableStream) from a POST handler; use ReadableStream with controller.enqueue() for each token chunk.
  • Client reader: use response.body.getReader() with a pull-based while (true) { reader.read() } loop and TextDecoder for UTF-8 decoding.
  • Abort handling: pass an AbortSignal to fetch() on the client and to the LLM SDK on the server via req.signal to cancel both sides cleanly.
  • Backpressure: the Web Streams pull model naturally throttles the producer; custom QueuingStrategy is rarely needed for token streams.
  • Vercel AI SDK: streamText() + toDataStreamResponse() on the server paired with useChat on the client eliminates most boilerplate and adds structured metadata support.
  • Error resilience: always use try/finally to release the reader lock, check response.ok before reading, and handle mid-stream drops gracefully.

常见问题解答

「逐令牌流式传输人工智能响应」课时是免费的吗？

是的 — 「逐令牌流式传输人工智能响应」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Next.js 15 Fullstack (App Router + Server Actions) 课程的其余内容，请升级到 CoddyKit PRO。 Next.js 15 Fullstack (App Router + Server Actions) 课程共包含 4 节课。

「逐令牌流式传输人工智能响应」这节课中我会学到什么？

使用具备背压感知能力的读取器和中止处理，将 LLM 补全内容流式传输到界面。 你通过在浏览器中直接运行的动手代码来练习 Next.js 15 Fullstack (App Router + Server Actions)，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Next.js 15 Fullstack (App Router + Server Actions) 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Next.js 15 Fullstack (App Router + Server Actions) 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 3 节课，共 4 节。

「逐令牌流式传输人工智能响应」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Next.js 15 Fullstack (App Router + Server Actions) 课中编写并运行代码吗？

能。每节 Next.js 15 Fullstack (App Router + Server Actions) 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

  1. 路由处理程序中的服务器发送事件
  2. 在无服务器环境中集成 WebSocket 服务
  3. 逐令牌流式传输人工智能响应
  4. 在线状态、光标与实时协作状态
← 返回 Next.js 15 Fullstack (App Router + Server Actions)