The idea
A 400-token answer can take 5–10 seconds to generate. Waiting for all of it feels broken; showing words as they arrive feels fast. Streaming APIs return a sequence of events (text deltas, tool call pieces, a final usage event) over one connection.
Your backend relays these to the browser, usually with Server-Sent Events. The UI appends text as it arrives. Handle cancellation (user closes the tab, so stop the upstream call), errors mid-stream and the final usage event for cost logging.
Streaming changes how you validate: you cannot check the full output before showing it. For structured outputs, stream to a progress UI and validate at the end.
It is the single biggest improvement to perceived speed in chat-style products.
Words to know
- DeltaA small piece of new text in a stream.
- SSEServer-Sent Events: one-way HTTP streaming to the browser.
Try it without code
Compare feel
- Ask a chat app for a long answer and watch it stream.
- Imagine the same wait with a spinner and no text.
Use it in code
Stream from the model to the browser with FastAPI and SSE.
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import anthropic, json
app = FastAPI()
client = anthropic.AsyncAnthropic()
@app.get("/chat")
async def chat(q: str):
async def events():
async with client.messages.stream(model="claude-sonnet-5-5", max_tokens=800,
messages=[{"role": "user", "content": q}]) as stream:
async for text in stream.text_stream:
yield f"data: {json.dumps({'text': text})}\n\n"
final = await stream.get_final_message()
yield f"data: {json.dumps({'done': True, 'usage': final.usage.model_dump()})}\n\n"
return StreamingResponse(events(), media_type="text/event-stream")Stream answers into a simple HTML page with EventSource and log time to first token.
Common mistakes
- Proxies buffering the stream (disable buffering).
- Not cancelling upstream when the user disconnects.