Streaming Responses

Your export endpoint queries the database, builds a 200,000-row report, and takes four seconds. The handler returns NextResponse.json(report), and the browser shows a spinner for the entire four seconds — because HTTP status and headers cannot be sent until you return a Response, and the body cannot start until it has been fully built. The work is done server-side in the first 400 ms of useful compute; the rest is serialization nobody can see.

A streaming route handler inverts that timeline. You return a Response immediately whose body is a ReadableStream, and every chunk leaves the server the moment it is produced. Time-to-first-byte collapses from four seconds to tens of milliseconds, while total transfer time stays the same. Streaming does not make the work faster — it stops making the user wait for work that does not concern them yet.

The API is the web standard one, so nothing Next-specific is required to produce bytes:

// app/api/export/route.ts
import { NextRequest, NextResponse } from "next/server";

export async function GET(request: NextRequest) {
  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();
      try {
        for await (const row of readOrdersInBatches(500)) {
          controller.enqueue(encoder.encode(JSON.stringify(row) + "\n"));
        }
        controller.close();
      } catch (err) {
        controller.error(err);
      }
    },
    cancel() {
      // the browser closed the tab: stop the producer here
    },
  });

  return new NextResponse(stream, {
    headers: {
      "Content-Type": "application/x-ndjson",
      "Cache-Control": "no-store",
    },
  });
}

The client never calls a special API: fetch() exposes response.body, a ReadableStream, and you pipe it — to a file via the Streams API, through a TextDecoderStream into a growing string, or out to a UI log via an async iterator:

const res = await fetch("/api/export");
const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  appendToLog(value);            // rows appear as they are produced
}

Two version details matter in interviews. First, in Next.js 14 a GET route handler that never touches the request was cached at build time by default; Next.js 15 removed that default — handlers run per request unless you cache explicitly. Second, reading the request (request.cookies, request.headers, request.signal) has always forced per-request behavior. A streaming handler is inherently dynamic: there is no static answer to "what does this response contain over its lifetime."

Backpressure is the part people skip. A ReadableStream with a sink has a default queuing strategy of one chunk (highWaterMark: 1), and the runtime calls pull(controller) again only after the consumer drains the queue. If your producer is an infinite loop in start(), you race ahead of a slow client and every unread chunk sits in process memory — thousands of rows in, a mobile client on a train tunnel, and the Node process is your buffer. Move the producer into pull() and each call produces one batch; the stream machinery then throttles production to consumption for free. If you must keep start(), check controller.desiredSize before enqueuing.

The numbers, for the interview: a 4-second buffered response has TTFB ≈ 4000 ms and FCP ≈ 4000 ms. The same handler streaming has TTFB ≈ 20–80 ms, first visible chunk under 200 ms, and the final byte still at 4000 ms. For dashboards, exports, AI-token feeds, and live search results, TTFB — not total time — is what the user experiences as "instant."

The report export streams once and closes. The harder case is a live feed: an admin dashboard that wants new orders as they are committed. The naive design — poll /api/orders every five seconds — spends one request per client per interval to learn that nothing changed 95% of the time, and still shows data up to five seconds late. Server-Sent Events (SSE) give you a persistent HTTP connection that the server writes to whenever something happens, with none of the handshake, upgrade, and protocol machinery of WebSockets.

SSE is a content type, not a framework feature: text/event-stream with a body of frames separated by blank lines.

// app/api/orders/live/route.ts
import { NextRequest, NextResponse } from "next/server";
import { orderEvents } from "@/lib/bus";

export const dynamic = "force-dynamic";

export async function GET(request: NextRequest) {
  let cleanup = () => {};

  const stream = new ReadableStream({
    start(controller) {
      const encoder = new TextEncoder();
      const frame = (data: unknown, id?: string, event?: string) =>
        controller.enqueue(
          encoder.encode(
            `${id ? `id: ${id}\n` : ""}${event ? `event: ${event}\n` : ""}data: ${JSON.stringify(data)}\n\n`
          )
        );

      const unsubscribe = orderEvents.subscribe((order) =>
        frame(order, String(order.seq), "order")
      );
      const heartbeat = setInterval(
        () => controller.enqueue(encoder.encode(": ping\n\n")),
        15_000
      );

      cleanup = () => {
        clearInterval(heartbeat);
        unsubscribe();
        try { controller.close(); } catch {}
      };
      request.signal.addEventListener("abort", cleanup);
    },
    cancel: cleanup,
  });

  return new NextResponse(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache, no-transform",
      "X-Accel-Buffering": "no",
    },
  });
}

Read the wire format precisely, because debugging it means reading raw bytes. Each frame carries named fields — data: (required, repeatable), event:, id:, retry: — and a frame ends at the first blank line. Lines starting with : are comments; that is what the heartbeat is, and it exists because every hop between you and the client (load balancer, NAT, corporate proxy) kills idle connections, typically after 30–120 seconds. Send a comment every 10–30 seconds and the connection lives.

The client side is one browser primitive:

const es = new EventSource("/api/orders/live");   // cookies ride along automatically
es.addEventListener("order", (e) => addRow(JSON.parse(e.data)));
es.onerror = () => es.close();                   // EventSource reconnects by itself

EventSource reconnects on its own and sends an Last-Event-ID header with the last id: it saw — which is exactly why you attach monotonic ids on the server: a client that dropped for eight seconds resumes where it left off instead of guessing. The trade-off you cannot skip: EventSource cannot set custom headers, so token-in-a-header authentication is unavailable — session cookies or query-string tokens are the options, and query strings leak into logs.

Two production constraints decide whether SSE fits. A long-lived connection per user ties up a slot for the connection's lifetime — fine on a Node server or container (thousands of connections per instance with epoll), awkward on serverless platforms where each instance traditionally served few concurrent functions, though AWS Lambda response streaming and Vercel's streaming functions changed the economics. And with more than one server instance, an event published on instance A reaches only connections held by A — real fan-out needs a pub/sub layer (Redis, a queue) so every instance hears every event.

Choose SSE when updates flow server → client only and arrive as discrete events. Choose WebSockets when the client also sends messages continuously (chat, multiplayer cursors, collaborative editing); in the App Router a WebSocket server still means a custom Node server or a separate service, because route handlers own the HTTP request/response cycle and cannot hold an upgraded socket.

Streaming introduces one hard rule that catches everyone once: the HTTP status line and headers are committed the moment you return the Response. A buffered handler can try its work and catch by returning 500; a streaming handler has already sent 200 OK before row one exists. An exception thrown mid-stream — a query that dies at row 150,000 — cannot become a 500. controller.error(err) merely destroys the connection, and the client sees a truncated body or a generic TypeError: network error from fetch, with no way to tell "the data was cut" from "the wifi dropped."

The professional pattern is in-band errors: keep the stream open and emit a terminal frame the client understands.

try {
  for await (const row of readOrdersInBatches()) {
    controller.enqueue(encoder.encode(JSON.stringify({ row }) + "\n"));
  }
  controller.enqueue(encoder.encode(JSON.stringify({ done: true }) + "\n"));
} catch (err) {
  controller.enqueue(
    encoder.encode(JSON.stringify({ error: "export failed", at: Date.now() }) + "\n")
  );
} finally {
    controller.close();
}

Client disconnect is the mirror image, and it is not hypothetical: users close tabs. When the consumer goes away, the runtime calls your cancel() sink method, and request.signal fires its abort event. If your producer keeps running — an unbounded loop, a DB cursor never closed, an AI generation still billing tokens — you are paying for bytes into a void. Hook the abort to teardown: close the cursor, unsubscribe from the bus, AbortController.abort() any outbound fetch. In a serverless deployment this also keeps your function from outliving its useful work.

The second classic failure is a stream that works in next dev on your laptop and arrives in one burst in production. Your handler is fine; something between it and the client is holding chunks. Three suspects, in the order they actually bite:

Verify with the Network tab: a healthy stream shows the response's time waterfall spread over seconds — first byte early, last byte late. If the whole body lands in one spike at the end, a buffer is eating your flushes. The same test from a terminal: curl -N http://localhost:3000/api/export prints rows as they arrive (-N disables curl's own buffering).

Two more boundaries worth knowing before an interviewer asks. Keep-alive mid-stream: platforms cut responses that send no bytes for a while — an SSE heartbeat and a long-running export's periodic progress frame both double as proof of life. And the payload ceiling: a stream has no single body-size limit, but each enqueue becomes a chunk on the wire; tiny chunks (one JSON object of 30 bytes each) turn into syscall and header overhead, so batch to at least a few KB per flush. The right mental model: a streaming route handler is a tiny protocol you design — status is fixed, errors are messages, the client is parsing, and teardown is part of the contract. Everything you learned about webhook idempotency in Verifying Webhook Signatures applies one level up: your consumer must tolerate duplicates and gaps, because a reconnecting client will ask twice.