The idea
Logs record what happened, metrics count and time things, error tracking groups exceptions with stack traces. Together they tell you that something is wrong, where, and why.
Write structured logs (JSON) with a request ID on every line so you can follow one request across services, plus user, tenant and feature fields. Expose metrics (request rate, errors, latency, tokens, cost) for dashboards and alerts. Send exceptions to an error tracker like Sentry with context.
Be careful with content: do not log full prompts, documents or personal data by default; log IDs and sizes, and keep sensitive traces in a restricted system.
You cannot operate what you cannot observe, and AI apps fail in more ways than normal ones.
Words to know
- Structured logA log line as key-value data rather than free text.
- Correlation IDAn ID that links all logs of one request.
Try it without code
Follow one request
- Make one request to your app.
- Try to find every log line it produced.
- If you cannot, you need request IDs.
Use it in code
JSON logs with a request ID middleware, plus Sentry.
# pip install structlog sentry-sdk[fastapi]
import uuid, structlog, sentry_sdk
from fastapi import FastAPI, Request
sentry_sdk.init(dsn="load-from-env", traces_sample_rate=0.1, send_default_pii=False)
structlog.configure(processors=[structlog.contextvars.merge_contextvars,
structlog.processors.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.JSONRenderer()])
log = structlog.get_logger()
app = FastAPI()
@app.middleware("http")
async def request_id(request: Request, call_next):
rid = request.headers.get("x-request-id", uuid.uuid4().hex)
structlog.contextvars.bind_contextvars(request_id=rid, path=request.url.path)
response = await call_next(request)
response.headers["x-request-id"] = rid
log.info("request_done", status=response.status_code)
structlog.contextvars.clear_contextvars()
return responseAdd structured logging with request IDs and Sentry to your API, then trigger an error and find it end to end.
Common mistakes
- Logging full prompts with personal data.
- print() statements as your only logging.