Area 08 · Backend · Concept 04 of 20

Streaming with SSE

Server-Sent Events push tokens to the browser over one HTTP connection.

01–10 · The basePhase 0 + 5 · Weeks 1–2, 18–20≈ 1 hour

The idea

Server-Sent Events is a simple standard for one-way streaming from server to browser over a normal HTTP response. The server sends lines like data: {...} followed by a blank line, and keeps the connection open. The browser’s EventSource API (or fetch with a stream reader) receives each event as it arrives.

It is the usual way to stream LLM tokens to a web UI: it works through most proxies, reconnects automatically, and needs no extra protocol. Send small JSON events: text deltas, status updates ("searching documents"), citations, and a final done event with usage.

Turn off proxy buffering, send periodic comments as heartbeats on long streams, and stop generation when the client disconnects.

Why it matters

Streaming is expected in chat UIs, and SSE is the simplest robust way to do it.

Words to know

  • EventSourceBrowser API for consuming SSE.
  • text/event-streamThe content type for SSE responses.

Try it without code

Watch an SSE stream

  1. Run the server below.
  2. Run: curl -N "localhost:8000/stream?q=hello"
  3. Watch events arrive one by one.

Use it in code

SSE with status events, text deltas and a done event; stops on disconnect.

python
import json, anthropic
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse

app = FastAPI()
client = anthropic.AsyncAnthropic()

def sse(obj): return f"data: {json.dumps(obj)}\n\n"

@app.get("/stream")
async def stream(q: str, request: Request):
    async def gen():
        yield sse({"type": "status", "text": "Thinking..."})
        async with client.messages.stream(model="claude-haiku-5-5", max_tokens=400,
                                          messages=[{"role": "user", "content": q}]) as s:
            async for text in s.text_stream:
                if await request.is_disconnected():
                    return
                yield sse({"type": "delta", "text": text})
        yield sse({"type": "done"})
    return StreamingResponse(gen(), media_type="text/event-stream",
                             headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
PRACTICE · TICK IT AFTER THIS

Build a tiny HTML page that renders the stream with fetch + ReadableStream and shows status events above the answer.

Common mistakes

  • Proxies buffering the whole response.
  • Continuing to pay for generation after the user left.

Go deeper