GuidesAI development

Tutorial: streaming an AI response in a web interface

By the Banzai Labs teamUpdated on 2 min read

No streaming: wait for it allStreaming: words arrive
Without streaming, the user waits for the whole answer. With it, words appear as they come.

1. Server side

An endpoint that returns a text stream (example in Astro route format, easy to port to Express or Next.js):

import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic();

export async function POST({ request }: { request: Request }) {
  const { question } = await request.json();
  const stream = client.messages.stream({
    model: process.env.CLAUDE_MODEL!,
    max_tokens: 1024,
    messages: [{ role: 'user', content: question }],
  });
  const enc = new TextEncoder();
  const body = new ReadableStream({
    start(ctrl) {
      stream.on('text', (t) => ctrl.enqueue(enc.encode(t)));
      stream.on('end', () => ctrl.close());
      stream.on('error', (e) => ctrl.error(e));
    },
  });
  return new Response(body, { headers: { 'Content-Type': 'text/plain; charset=utf-8' } });
}

2. Browser side

const res = await fetch('/api/ask', { method: 'POST', body: JSON.stringify({ question }) });
const reader = res.body!.getReader();
const dec = new TextDecoder();
for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  output.textContent += dec.decode(value, { stream: true });
}

3. The details that matter

  • An indicatorA blinking cursor while the stream is open.
  • The Stop buttonAn AbortController, wired to the server-side stream.
  • MarkdownRendered at the end, or block by block, to avoid jumps.

Before this: calling the Claude API from Node.js.

Frequently asked questions

Does streaming cost more?
No. The number of tokens is the same; only the way you receive them changes.
Do I need WebSockets?
No. An HTTP response read in chunks is enough, or Server-Sent Events.
How do I stop an answer in progress?
In the browser, an AbortController cancels the request; on the server, you stop the SDK stream when the connection closes.

Read next

All guides