The idea
Server-Sent Events is a simple standard for one-way streaming from server to browser over a normal HTTP response. The server sends lines like data: {...} followed by a blank line, and keeps the connection open. The browser’s EventSource API (or fetch with a stream reader) receives each event as it arrives.
It is the usual way to stream LLM tokens to a web UI: it works through most proxies, reconnects automatically, and needs no extra protocol. Send small JSON events: text deltas, status updates ("searching documents"), citations, and a final done event with usage.
Turn off proxy buffering, send periodic comments as heartbeats on long streams, and stop generation when the client disconnects.
Streaming is expected in chat UIs, and SSE is the simplest robust way to do it.
Words to know
- EventSourceBrowser API for consuming SSE.
- text/event-streamThe content type for SSE responses.
Try it without code
Watch an SSE stream
- Run the server below.
- Run: curl -N "localhost:8000/stream?q=hello"
- Watch events arrive one by one.
Use it in code
SSE with status events, text deltas and a done event; stops on disconnect.
import json, anthropic
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
app = FastAPI()
client = anthropic.AsyncAnthropic()
def sse(obj): return f"data: {json.dumps(obj)}\n\n"
@app.get("/stream")
async def stream(q: str, request: Request):
async def gen():
yield sse({"type": "status", "text": "Thinking..."})
async with client.messages.stream(model="claude-haiku-5-5", max_tokens=400,
messages=[{"role": "user", "content": q}]) as s:
async for text in s.text_stream:
if await request.is_disconnected():
return
yield sse({"type": "delta", "text": text})
yield sse({"type": "done"})
return StreamingResponse(gen(), media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})Build a tiny HTML page that renders the stream with fetch + ReadableStream and shows status events above the answer.
Common mistakes
- Proxies buffering the whole response.
- Continuing to pay for generation after the user left.