Area 04 · Evals · Concept 15 of 20

Tracing

Record every step of a request (prompts, tool calls, time, tokens) so you can debug it later.

11–20 · Production levelPhase 4 · Weeks 15–17≈ 1 hour

The idea

When a user says "it gave me a wrong answer yesterday", you need to see exactly what happened: the prompt sent, the chunks retrieved, each tool call and result, each model output, timings, tokens and cost. A trace records all of that as a tree of spans for one request.

Instrument at the boundaries: each model call, each retrieval, each tool call, plus a root span per user request with user and session IDs. Most observability tools provide decorators or SDK wrappers that do this automatically.

Traces are also the raw material for evals: you sample them for error analysis, label them, and turn failures into new golden cases.

Why it matters

You cannot debug or improve what you cannot see. Tracing is the first thing to add in production.

Picture it

Words to know

  • SpanOne timed operation inside a trace.
  • TraceThe tree of spans for one request.

Try it without code

Trace by printing

  1. Add a print with timestamp and token counts before and after each step of your RAG pipeline.
  2. Run one question and read the log top to bottom.

Use it in code

A tiny tracer as a context manager, to understand what tools like Langfuse do.

python
import time, json, uuid
from contextlib import contextmanager

TRACE = []

@contextmanager
def span(name, **attrs):
    s = {"id": uuid.uuid4().hex[:8], "name": name, "start": time.time(), **attrs}
    try:
        yield s
    finally:
        s["ms"] = round((time.time() - s.pop("start")) * 1000)
        TRACE.append(s)

with span("request", user="u1"):
    with span("retrieve") as s: s["chunks"] = 20; time.sleep(0.1)
    with span("generate") as s: s["tokens_in"] = 1900; time.sleep(0.2)
print(json.dumps(TRACE, indent=2))
PRACTICE · TICK IT AFTER THIS

Add spans for every stage of your RAG app and print the slowest stage for 10 questions.

Common mistakes

  • Logging prompts with personal data and no redaction.
  • No request ID linking logs from different services.

Go deeper