0Pricing
Next.js 15 Fullstack (App Router + Server Actions) · Leçon

Diffuser les réponses de l’IA jeton par jeton

Diffusez les complétions de LLM vers l’interface avec des lecteurs gérant la contre-pression et l’interruption.

Diffuser les réponses de l’IA jeton par jeton est une leçon Next.js 15 Fullstack (App Router + Server Actions) gratuite sur CoddyKit. Ceci est la leçon 3 sur 4. Tu peux lire la leçon complète ci-dessous gratuitement — puis la pratiquer en direct dans le navigateur avec un éditeur de code intégré et un tuteur IA 24/7. Elle fait partie du parcours d'apprentissage Next.js 15 Fullstack (App Router + Server Actions), et ta progression se synchronise sur le web et l'application CoddyKit. Le cours Next.js 15 Fullstack (App Router + Server Actions) comprend 4 leçons au total.

Certaines parties de cette leçon n'ont pas encore été traduites et s'affichent en anglais.

Why Stream AI Responses?

When you call a large language model (LLM) like GPT-4 or Claude, the model generates tokens one by one. A typical response may take 5–20 seconds to complete. If you wait for the full response before sending anything to the client, the user stares at a blank screen the entire time.

Streaming solves this: you pipe each token to the browser as it is produced, creating the familiar "typewriter" effect used by ChatGPT, Claude.ai, and Gemini.

  • Perceived latency drops from time-to-full-response to time-to-first-token (often under 300 ms).
  • Users can start reading and even abort early if the answer is already clear.
  • Server memory stays flat — you never buffer the whole response.

In Next.js 15 the primitives you need are ReadableStream, the Web Streams API, and StreamingTextResponse (or a plain Response with a stream body).

How LLM SDKs Expose Streams

Most LLM SDKs return an async iterable or a ReadableStream when you pass stream: true. The Vercel AI SDK unifies these under a single interface.

With the official OpenAI SDK you receive a stream of ChatCompletionChunk objects. Each chunk carries a delta.content string that may be one token, a few characters, or an empty string at the end.

  • OpenAI SDK: openai.chat.completions.create({ stream: true }) returns an AsyncIterable.
  • Vercel AI SDK: streamText() returns a result with result.toDataStreamResponse() ready for Next.js Route Handlers.
  • Anthropic SDK: client.messages.stream() returns an async iterable of MessageStreamEvent.

Regardless of the SDK, the pattern is the same: iterate over chunks, encode each piece, and enqueue it into a ReadableStream that becomes the HTTP response body.

Creating a Streaming Route Handler

A Next.js 15 Route Handler returns a standard Response. You can pass a ReadableStream as the body to stream data to the client. The ReadableStream constructor accepts a start(controller) callback where you push chunks with controller.enqueue() and signal completion with controller.close().

Below is a minimal Route Handler at app/api/chat/route.ts that streams an OpenAI completion token by token.

// app/api/chat/route.ts
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create({
    model: 'gpt-4o-mini',
    stream: true,
    messages: [{ role: 'user', content: prompt }],
  });

  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          controller.enqueue(encoder.encode(text));
        }
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Transfer-Encoding': 'chunked',
    },
  });
}

Reading the Stream on the Client

On the browser side you use the Fetch API together with a ReadableStreamDefaultReader to consume the stream incrementally. The key steps are:

  1. Call fetch() — do not await response.json() (that buffers everything).
  2. Get the reader: response.body!.getReader().
  3. Loop with reader.read() until done === true.
  4. Decode each Uint8Array chunk with TextDecoder and append to state.

This is called a pull-based read loop. You pull the next chunk only after you have processed the previous one, giving you natural backpressure.

// Standalone client utility — no React dependency
async function streamCompletion(
  prompt: string,
  onChunk: (text: string) => void
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    const text = decoder.decode(value, { stream: true });
    onChunk(text);
  }
}

// Usage example (TypeScript, no browser DOM required for the logic):
// streamCompletion('Hello!', (chunk) => process.stdout.write(chunk));

Abort Handling with AbortController

Users frequently stop a generation mid-stream. Without abort handling, the server keeps calling the LLM and the client leaks a reader that never closes.

The fix is AbortController. You pass its signal to fetch(). When you call controller.abort(), the fetch throws an AbortError and the browser cancels the underlying TCP read, which in turn triggers backpressure cancellation upstream.

  • Always wrap the read loop in a try/finally block so reader.releaseLock() is always called.
  • On the server, pass the request's signal to the OpenAI SDK so the LLM call itself is cancelled — this saves tokens and money.
// Client-side abort-aware stream reader
async function streamWithAbort(
  prompt: string,
  onChunk: (text: string) => void,
  signal: AbortSignal
): Promise<void> {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ prompt }),
    signal, // <-- pass AbortSignal to fetch
  });

  if (!response.ok || !response.body) {
    throw new Error(`HTTP ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  try {
    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(decoder.decode(value, { stream: true }));
    }
  } finally {
    reader.releaseLock(); // always release even on abort
  }
}

Propagating Abort to the LLM on the Server

When the client aborts, the fetch connection closes. Next.js 15 Route Handlers expose req.signal — a native AbortSignal that fires when the client disconnects. Pass this signal to the LLM SDK to cancel the upstream API call immediately.

Inside the ReadableStream constructor you can also implement a cancel() method that cleans up any ongoing work when the stream is cancelled by the consumer.

// app/api/chat/route.ts — with server-side abort propagation
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };

  const completion = await openai.chat.completions.create(
    {
      model: 'gpt-4o-mini',
      stream: true,
      messages: [{ role: 'user', content: prompt }],
    },
    { signal: req.signal } // propagate client disconnect signal
  );

  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of completion) {
          if (req.signal.aborted) break;
          const text = chunk.choices[0]?.delta?.content ?? '';
          if (text) controller.enqueue(encoder.encode(text));
        }
      } catch (err) {
        if ((err as Error).name !== 'AbortError') throw err;
      } finally {
        controller.close();
      }
    },
    cancel() {
      // Called when the consumer (browser) cancels the stream
      completion.controller.abort();
    },
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

Using the Vercel AI SDK for Simpler Streaming

The Vercel AI SDK (ai package) removes most of the boilerplate. streamText() handles the stream construction, abort propagation, and error boundaries for you. It also adds a structured data stream format that the client-side useChat hook can parse.

The result object exposes result.toDataStreamResponse() which returns a fully-formed Response ready to return from your Route Handler.

// app/api/chat/route.ts — Vercel AI SDK
import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

export async function POST(req: Request): Promise<Response> {
  const { messages } = await req.json();

  const result = streamText({
    model: openai('gpt-4o-mini'),
    messages,
    abortSignal: req.signal, // automatic abort propagation
  });

  // toDataStreamResponse() streams tokens in Vercel AI data-stream format
  return result.toDataStreamResponse();
}

The useChat Hook on the Client

When the server uses the Vercel AI SDK data-stream format, the companion useChat hook on the client handles everything: fetching, reading the stream, appending tokens, and exposing an isLoading flag plus a stop() function for abort.

This is the recommended pattern for chat UIs in Next.js 15 App Router projects. The hook is framework-agnostic enough to work in both Client Components and Server Components that hydrate client islands.

// app/chat/page.tsx — Client Component using useChat
'use client';

import { useChat } from 'ai/react';

export default function ChatPage() {
  const { messages, input, handleInputChange, handleSubmit, isLoading, stop } =
    useChat({ api: '/api/chat' });

  return (
    <main style={{ maxWidth: 600, margin: '0 auto', padding: 24 }}>
      <ul>
        {messages.map((m) => (
          <li key={m.id}>
            <strong>{m.role}:</strong> {m.content}
          </li>
        ))}
      </ul>
      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} placeholder="Ask anything..." />
        <button type="submit" disabled={isLoading}>Send</button>
        {isLoading && (
          <button type="button" onClick={stop}>Stop</button>
        )}
      </form>
    </main>
  );
}

Backpressure: Why It Matters

Backpressure is the mechanism that prevents a fast producer (the LLM API) from overwhelming a slow consumer (the browser rendering pipeline or a slow network).

In the Web Streams API, backpressure is built in:

  • Each call to reader.read() waits for the consumer to be ready before requesting the next chunk from the underlying source.
  • The ReadableStream internal queue holds a limited number of chunks (controlled by QueuingStrategy). When the queue is full, the producer is paused automatically.
  • If you use for await...of on a stream, the JavaScript runtime handles pull-pacing for you — you get one chunk per iteration, naturally throttled.

You rarely need to configure a custom QueuingStrategy for AI token streams because LLM responses are slow enough that the default high-watermark (1 chunk) is sufficient.

// Demonstrating a custom ByteLengthQueuingStrategy (illustrative)
const strategy = new ByteLengthQueuingStrategy({ highWaterMark: 1024 }); // 1 KB buffer

const stream = new ReadableStream(
  {
    start(controller) {
      // producer: enqueue only when consumer is ready
      controller.enqueue(new TextEncoder().encode('Hello '));
      controller.enqueue(new TextEncoder().encode('world!'));
      controller.close();
    },
  },
  strategy
);

// Consumer: pull-based, respects backpressure
const reader = stream.getReader();
async function drain() {
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    console.log(new TextDecoder().decode(value));
  }
}
drain();

Server-Sent Events vs Raw Streaming

There are two common wire formats for streaming AI responses to the browser:

  • Raw byte stream (Content-Type: text/plain): the simplest approach. Each chunk is raw UTF-8 text. Used in the earlier examples. Works well for simple prose but carries no metadata (token counts, finish reasons, etc.).
  • Server-Sent Events (SSE) (Content-Type: text/event-stream): a standardised line-based format. Each event is data: <payload>\n\n. Browsers have a built-in EventSource API, but it only supports GET. For POST-based chat you use a custom SSE parser on top of fetch.

The Vercel AI SDK data-stream format is an SSE-like protocol that encodes token deltas, tool calls, and metadata as structured lines. Use raw streaming for simple use-cases; use the AI SDK protocol when you need structured events.

// app/api/chat-sse/route.ts — manual SSE format over POST
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

export async function POST(req: Request): Promise<Response> {
  const { prompt } = await req.json() as { prompt: string };
  const encoder = new TextEncoder();

  const sseStream = new ReadableStream({
    async start(controller) {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4o-mini',
        stream: true,
        messages: [{ role: 'user', content: prompt }],
      });

      for await (const chunk of completion) {
        const text = chunk.choices[0]?.delta?.content ?? '';
        if (text) {
          // SSE line: data: <payload>\n\n
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ text })}\n\n`));
        }
      }
      controller.enqueue(encoder.encode('data: [DONE]\n\n'));
      controller.close();
    },
  });

  return new Response(sseStream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      Connection: 'keep-alive',
    },
  });
}

Error Handling and Finish Reasons

Streaming introduces new error scenarios that do not exist with a single-shot response:

  • Mid-stream network drop: the read loop throws; catch it and show a "Connection lost" UI.
  • LLM rate-limit (429): the HTTP response itself is non-2xx; check response.ok before reading the body.
  • Content filter stop: the stream ends normally but the last chunk's finish_reason is 'content_filter' — you must read finish reasons out-of-band or from a final SSE event.
  • Token limit reached: finish_reason === 'length' — inform the user the response was truncated.

Always implement a try/catch around the read loop and display graceful fallback UI. Never leave the stream reader locked on error.

// Robust client-side stream reader with error handling
async function safeStream(
  prompt: string,
  onChunk: (t: string) => void,
  onError: (msg: string) => void,
  signal: AbortSignal
): Promise<void> {
  let reader: ReadableStreamDefaultReader<Uint8Array> | null = null;
  try {
    const res = await fetch('/api/chat', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
      signal,
    });

    if (!res.ok) {
      onError(`Server error: ${res.status} ${res.statusText}`);
      return;
    }

    reader = res.body!.getReader();
    const dec = new TextDecoder();

    while (true) {
      const { value, done } = await reader.read();
      if (done) break;
      onChunk(dec.decode(value, { stream: true }));
    }
  } catch (err) {
    if ((err as Error).name === 'AbortError') return; // user-initiated, silent
    onError((err as Error).message);
  } finally {
    reader?.releaseLock();
  }
}

Knowledge Check: Abort Handling

Consider a Next.js 15 Route Handler that streams an OpenAI completion. A user clicks Stop in the UI, which calls controller.abort() on an AbortController whose signal was passed to fetch(). Which additional step is most critical to prevent wasted LLM tokens and unnecessary server compute?

Lesson Recap

In this lesson you learned how to stream LLM completions token-by-token in a Next.js 15 App Router application:

  • Why stream: reduces perceived latency from seconds to milliseconds by sending the first token before the full response is ready.
  • Route Handler: return a new Response(readableStream) from a POST handler; use ReadableStream with controller.enqueue() for each token chunk.
  • Client reader: use response.body.getReader() with a pull-based while (true) { reader.read() } loop and TextDecoder for UTF-8 decoding.
  • Abort handling: pass an AbortSignal to fetch() on the client and to the LLM SDK on the server via req.signal to cancel both sides cleanly.
  • Backpressure: the Web Streams pull model naturally throttles the producer; custom QueuingStrategy is rarely needed for token streams.
  • Vercel AI SDK: streamText() + toDataStreamResponse() on the server paired with useChat on the client eliminates most boilerplate and adds structured metadata support.
  • Error resilience: always use try/finally to release the reader lock, check response.ok before reading, and handle mid-stream drops gracefully.

Questions Fréquemment Posées

La leçon « Diffuser les réponses de l’IA jeton par jeton » est-elle gratuite ?

Oui — le texte complet de « Diffuser les réponses de l’IA jeton par jeton » est gratuit à lire ici sur le web. Pour la pratiquer de manière interactive (un éditeur de code intégré et un tuteur IA 24/7) et déverrouiller le reste du cours Next.js 15 Fullstack (App Router + Server Actions), passe à CoddyKit PRO. Le cours Next.js 15 Fullstack (App Router + Server Actions) comprend 4 leçons au total.

Qu'est-ce que j'apprendrai dans « Diffuser les réponses de l’IA jeton par jeton » ?

Diffusez les complétions de LLM vers l’interface avec des lecteurs gérant la contre-pression et l’interruption. Tu pratiques Next.js 15 Fullstack (App Router + Server Actions) avec du code pratique que tu exécutes directement dans le navigateur, et un tuteur IA 24/7 répond à tes questions au fur et à mesure que tu avances dans la leçon.

Dois-je avoir de l'expérience pour commencer Next.js 15 Fullstack (App Router + Server Actions) ?

Aucune expérience préalable n'est requise. Next.js 15 Fullstack (App Router + Server Actions) sur CoddyKit est structuré pour les débutants jusqu'aux apprenants avancés, donc tu peux commencer ici ou depuis le début et avancer à ton rythme. Ceci est la leçon 3 sur 4.

Combien de temps prend la leçon « Diffuser les réponses de l’IA jeton par jeton » ?

La plupart des leçons CoddyKit prennent environ 5–10 minutes. Chacune est courte et interactive, tu progresses régulièrement et tu repiques exactement où tu t'es arrêté sur le web et l'app.

Peux-tu écrire et exécuter du code dans cette leçon Next.js 15 Fullstack (App Router + Server Actions) ?

Oui. Chaque leçon Next.js 15 Fullstack (App Router + Server Actions) inclut un éditeur de code intégré, tu écris et exécutes du vrai code directement dans ton navigateur et tu reçois des retours IA instantanés — aucune configuration locale requise.

Toutes les leçons de ce cours

  1. Événements envoyés par le serveur depuis les gestionnaires de routes
  2. Intégrer des services WebSocket dans un environnement sans serveur
  3. Diffuser les réponses de l’IA jeton par jeton
  4. Présence, curseurs et état de collaboration en direct
← Retour à Next.js 15 Fullstack (App Router + Server Actions)