0Pricing
Next.js 15 Fullstack (App Router + Server Actions) · Lección

Streaming de respuestas de IA token a token

Transmita las finalizaciones de LLM a la interfaz con lectores que gestionen la contrapresión y la cancelación.

Streaming de respuestas de IA token a token es una lección gratuita de Next.js 15 Fullstack (App Router + Server Actions) en CoddyKit. Esta es la lección 3 de 4. Puedes leer la lección completa abajo gratuitamente — luego la practicas en el navegador con un editor de código integrado y un tutor de IA 24/7. Forma parte de la ruta de aprendizaje de Next.js 15 Fullstack (App Router + Server Actions), y tu progreso se sincroniza en la web y la app de CoddyKit. El curso de Next.js 15 Fullstack (App Router + Server Actions) incluye 4 lecciones en total.

Partes de esta lección aún no han sido traducidas y se muestran en inglés.

Why Stream AI Responses?

When you call a large language model (LLM) like GPT-4 or Claude, the model generates tokens one by one. A typical response may take 5–20 seconds to complete. If you wait for the full response before sending anything to the client, the user stares at a blank screen the entire time.

Streaming solves this: you pipe each token to the browser as it is produced, creating the familiar "typewriter" effect used by ChatGPT, Claude.ai, and Gemini.

  • Perceived latency drops from time-to-full-response to time-to-first-token (often under 300 ms).
  • Users can start reading and even abort early if the answer is already clear.
  • Server memory stays flat — you never buffer the whole response.

In Next.js 15 the primitives you need are ReadableStream, the Web Streams API, and StreamingTextResponse (or a plain Response with a stream body).

How LLM SDKs Expose Streams

Most LLM SDKs return an async iterable or a ReadableStream when you pass stream: true. The Vercel AI SDK unifies these under a single interface.

With the official OpenAI SDK you receive a stream of ChatCompletionChunk objects. Each chunk carries a delta.content string that may be one token, a few characters, or an empty string at the end.

  • OpenAI SDK: openai.chat.completions.create({ stream: true }) returns an AsyncIterable.
  • Vercel AI SDK: streamText() returns a result with result.toDataStreamResponse() ready for Next.js Route Handlers.
  • Anthropic SDK: client.messages.stream() returns an async iterable of MessageStreamEvent.

Regardless of the SDK, the pattern is the same: iterate over chunks, encode each piece, and enqueue it into a ReadableStream that becomes the HTTP response body.

Creating a Streaming Route Handler

A Next.js 15 Route Handler returns a standard Response. You can pass a ReadableStream as the body to stream data to the client. The ReadableStream constructor accepts a start(controller) callback where you push chunks with controller.enqueue() and signal completion with controller.close().

Below is a minimal Route Handler at app/api/chat/route.ts that streams an OpenAI completion token by token.

// app/api/chat/route.ts
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    stream: true,
    messages: [{ role: 'user', content: prompt }],
  });

  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          controller.enqueue(encoder.encode(text));
        }
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Transfer-Encoding': 'chunked',
    },
  });
}

Reading the Stream on the Client

On the browser side you use the Fetch API together with a ReadableStreamDefaultReader to consume the stream incrementally. The key steps are:

  1. Call fetch() — do not await response.json() (that buffers everything).
  2. Get the reader: response.body!.getReader().
  3. Loop with reader.read() until done === true.
  4. Decode each Uint8Array chunk with TextDecoder and append to state.

This is called a pull-based read loop. You pull the next chunk only after you have processed the previous one, giving you natural backpressure.

// Standalone client utility — no React dependency
async function streamCompletion(
  prompt: string,
  onChunk: (text: string) => void
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    const text = decoder.decode(value, { stream: true });
    onChunk(text);
  }
}

// Usage example (TypeScript, no browser DOM required for the logic):
// streamCompletion('Hello!', (chunk) => process.stdout.write(chunk));

Abort Handling with AbortController

Users frequently stop a generation mid-stream. Without abort handling, the server keeps calling the LLM and the client leaks a reader that never closes.

The fix is AbortController. You pass its signal to fetch(). When you call controller.abort(), the fetch throws an AbortError and the browser cancels the underlying TCP read, which in turn triggers backpressure cancellation upstream.

  • Always wrap the read loop in a try/finally block so reader.releaseLock() is always called.
  • On the server, pass the request's signal to the OpenAI SDK so the LLM call itself is cancelled — this saves tokens and money.
// Client-side abort-aware stream reader
async function streamWithAbort(
  prompt: string,
  onChunk: (text: string) => void,
  signal: AbortSignal
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
    signal, // <-- pass AbortSignal to fetch
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(decoder.decode(value, { stream: true }));
    }
  } finally {
    reader.releaseLock(); // always release even on abort
  }
}

Propagating Abort to the LLM on the Server

When the client aborts, the fetch connection closes. Next.js 15 Route Handlers expose req.signal — a native AbortSignal that fires when the client disconnects. Pass this signal to the LLM SDK to cancel the upstream API call immediately.

Inside the ReadableStream constructor you can also implement a cancel() method that cleans up any ongoing work when the stream is cancelled by the consumer.

// app/api/chat/route.ts — with server-side abort propagation
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create(
    {
      model: 'gpt-4o-mini',
      stream: true,
      messages: [{ role: 'user', content: prompt }],
    },
    { signal: req.signal } // propagate client disconnect signal
  );

  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of completion) {
          if (req.signal.aborted) break;
          const text = chunk.choices[0]?.delta?.content ?? '';
          if (text) controller.enqueue(encoder.encode(text));
        }
      } catch (err) {
        if ((err as Error).name !== 'AbortError') throw err;
      } finally {
        controller.close();
      }
    },
    cancel() {
      // Called when the consumer (browser) cancels the stream
      completion.controller.abort();
    },
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

Using the Vercel AI SDK for Simpler Streaming

The Vercel AI SDK (ai package) removes most of the boilerplate. streamText() handles the stream construction, abort propagation, and error boundaries for you. It also adds a structured data stream format that the client-side useChat hook can parse.

The result object exposes result.toDataStreamResponse() which returns a fully-formed Response ready to return from your Route Handler.

// app/api/chat/route.ts — Vercel AI SDK
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request): Promise<Response> {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
    abortSignal: req.signal, // automatic abort propagation
  });

  // toDataStreamResponse() streams tokens in Vercel AI data-stream format
  return result.toDataStreamResponse();
}

The useChat Hook on the Client

When the server uses the Vercel AI SDK data-stream format, the companion useChat hook on the client handles everything: fetching, reading the stream, appending tokens, and exposing an isLoading flag plus a stop() function for abort.

This is the recommended pattern for chat UIs in Next.js 15 App Router projects. The hook is framework-agnostic enough to work in both Client Components and Server Components that hydrate client islands.

// app/chat/page.tsx — Client Component using useChat
'use client';

import { useChat } from 'ai/react';

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, stop } =
    useChat({ api: '/api/chat' });

  return (
    <main style={{ maxWidth: 600, margin: '0 auto', padding: 24 }}>
      <ul>
        {messages.map((m) => (
          <li key={m.id}>
            <strong>{m.role}:</strong> {m.content}
          </li>
        ))}
      </ul>
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Ask anything..." />
        <button type="submit" disabled={isLoading}>Send</button>
        {isLoading && (
          <button type="button" onClick={stop}>Stop</button>
        )}
      </form>
    </main>
  );
}

Backpressure: Why It Matters

Backpressure is the mechanism that prevents a fast producer (the LLM API) from overwhelming a slow consumer (the browser rendering pipeline or a slow network).

In the Web Streams API, backpressure is built in:

  • Each call to reader.read() waits for the consumer to be ready before requesting the next chunk from the underlying source.
  • The ReadableStream internal queue holds a limited number of chunks (controlled by QueuingStrategy). When the queue is full, the producer is paused automatically.
  • If you use for await...of on a stream, the JavaScript runtime handles pull-pacing for you — you get one chunk per iteration, naturally throttled.

You rarely need to configure a custom QueuingStrategy for AI token streams because LLM responses are slow enough that the default high-watermark (1 chunk) is sufficient.

// Demonstrating a custom ByteLengthQueuingStrategy (illustrative)
const strategy = new ByteLengthQueuingStrategy({ highWaterMark: 1024 }); // 1 KB buffer

const stream = new ReadableStream(
  {
    start(controller) {
      // producer: enqueue only when consumer is ready
      controller.enqueue(new TextEncoder().encode('Hello '));
      controller.enqueue(new TextEncoder().encode('world!'));
      controller.close();
    },
  },
  strategy
);

// Consumer: pull-based, respects backpressure
const reader = stream.getReader();
async function drain() {
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    console.log(new TextDecoder().decode(value));
  }
}
drain();

Server-Sent Events vs Raw Streaming

There are two common wire formats for streaming AI responses to the browser:

  • Raw byte stream (Content-Type: text/plain): the simplest approach. Each chunk is raw UTF-8 text. Used in the earlier examples. Works well for simple prose but carries no metadata (token counts, finish reasons, etc.).
  • Server-Sent Events (SSE) (Content-Type: text/event-stream): a standardised line-based format. Each event is data: <payload>\n\n. Browsers have a built-in EventSource API, but it only supports GET. For POST-based chat you use a custom SSE parser on top of fetch.

The Vercel AI SDK data-stream format is an SSE-like protocol that encodes token deltas, tool calls, and metadata as structured lines. Use raw streaming for simple use-cases; use the AI SDK protocol when you need structured events.

// app/api/chat-sse/route.ts — manual SSE format over POST
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };
  const encoder = new TextEncoder();

  const sseStream = new ReadableStream({
    async start(controller) {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4o-mini',
        stream: true,
        messages: [{ role: 'user', content: prompt }],
      });

      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          // SSE line: data: <payload>\n\n
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ text })}\n\n`));
        }
      }
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });

  return new Response(sseStream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

Error Handling and Finish Reasons

Streaming introduces new error scenarios that do not exist with a single-shot response:

  • Mid-stream network drop: the read loop throws; catch it and show a "Connection lost" UI.
  • LLM rate-limit (429): the HTTP response itself is non-2xx; check response.ok before reading the body.
  • Content filter stop: the stream ends normally but the last chunk's finish_reason is 'content_filter' — you must read finish reasons out-of-band or from a final SSE event.
  • Token limit reached: finish_reason === 'length' — inform the user the response was truncated.

Always implement a try/catch around the read loop and display graceful fallback UI. Never leave the stream reader locked on error.

// Robust client-side stream reader with error handling
async function safeStream(
  prompt: string,
  onChunk: (t: string) => void,
  onError: (msg: string) => void,
  signal: AbortSignal
): Promise<void> {
  let reader: ReadableStreamDefaultReader<Uint8Array> | null = null;
  try {
    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
      signal,
    });

    if (!res.ok) {
      onError(`Server error: ${res.status} ${res.statusText}`);
      return;
    }

    reader = res.body!.getReader();
    const dec = new TextDecoder();

    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(dec.decode(value, { stream: true }));
    }
  } catch (err) {
    if ((err as Error).name === 'AbortError') return; // user-initiated, silent
    onError((err as Error).message);
  } finally {
    reader?.releaseLock();
  }
}

Knowledge Check: Abort Handling

Consider a Next.js 15 Route Handler that streams an OpenAI completion. A user clicks Stop in the UI, which calls controller.abort() on an AbortController whose signal was passed to fetch(). Which additional step is most critical to prevent wasted LLM tokens and unnecessary server compute?

Lesson Recap

In this lesson you learned how to stream LLM completions token-by-token in a Next.js 15 App Router application:

  • Why stream: reduces perceived latency from seconds to milliseconds by sending the first token before the full response is ready.
  • Route Handler: return a new Response(readableStream) from a POST handler; use ReadableStream with controller.enqueue() for each token chunk.
  • Client reader: use response.body.getReader() with a pull-based while (true) { reader.read() } loop and TextDecoder for UTF-8 decoding.
  • Abort handling: pass an AbortSignal to fetch() on the client and to the LLM SDK on the server via req.signal to cancel both sides cleanly.
  • Backpressure: the Web Streams pull model naturally throttles the producer; custom QueuingStrategy is rarely needed for token streams.
  • Vercel AI SDK: streamText() + toDataStreamResponse() on the server paired with useChat on the client eliminates most boilerplate and adds structured metadata support.
  • Error resilience: always use try/finally to release the reader lock, check response.ok before reading, and handle mid-stream drops gracefully.

Preguntas frecuentes

¿La lección «Streaming de respuestas de IA token a token» es gratis?

Sí — el texto completo de «Streaming de respuestas de IA token a token» es gratis para leer aquí en la web. Para practicarla de forma interactiva (editor de código integrado y tutor de IA 24/7) y desbloquear el resto del curso de Next.js 15 Fullstack (App Router + Server Actions), actualiza a CoddyKit PRO. El curso de Next.js 15 Fullstack (App Router + Server Actions) incluye 4 lecciones en total.

¿Qué aprenderé en «Streaming de respuestas de IA token a token»?

Transmita las finalizaciones de LLM a la interfaz con lectores que gestionen la contrapresión y la cancelación. Practicas Next.js 15 Fullstack (App Router + Server Actions) con código real que ejecutas directamente en el navegador, y un tutor de IA 24/7 responde tus preguntas mientras trabajas en la lección.

¿Necesito experiencia previa para empezar Next.js 15 Fullstack (App Router + Server Actions)?

No se requiere experiencia previa. Next.js 15 Fullstack (App Router + Server Actions) en CoddyKit está estructurado para principiantes hasta estudiantes avanzados, así que puedes empezar aquí o desde el inicio y avanzar a tu ritmo. Esta es la lección 3 de 4.

¿Cuánto tiempo toma la lección «Streaming de respuestas de IA token a token»?

La mayoría de las lecciones de CoddyKit toman alrededor de 5–10 minutos. Cada una es compacta e interactiva, así que avanzas constantemente y retomas exactamente por donde dejaste en la web y la app.

¿Puedo escribir y ejecutar código en esta lección de Next.js 15 Fullstack (App Router + Server Actions)?

Sí. Cada lección de Next.js 15 Fullstack (App Router + Server Actions) incluye un editor de código integrado, así que escribes y ejecutas código real directamente en tu navegador y obtienes retroalimentación instantánea de IA — sin configuración local necesaria.

Todas las lecciones de este curso

  1. Eventos enviados por el servidor desde handlers de rutas
  2. Integración de servicios WebSocket en un entorno serverless
  3. Streaming de respuestas de IA token a token
  4. Presencia, cursores y estado de colaboración en tiempo real
← Volver a Next.js 15 Fullstack (App Router + Server Actions)