Area 04 · Evals · Concept 16 of 20

Langfuse

Open-source LLM observability: traces, prompt management, evals and cost tracking.

11–20 · Production levelPhase 4 · Weeks 15–17≈ 1.5 hours

The idea

Langfuse collects traces from your app and shows them in a web UI: each request’s spans, inputs and outputs, latency, token usage and cost. You can self-host it or use the managed cloud.

Beyond tracing it offers prompt management (versioned prompts fetched at runtime), datasets (golden sets) and experiments (run a dataset through a prompt version and score it), plus scores from judges, users or humans attached to traces.

Integration is light: an @observe decorator on your functions, or drop-in wrappers for popular SDKs, plus OpenTelemetry support.

Why it matters

It gives you tracing, cost tracking and eval tracking in one place without building dashboards yourself.

Words to know

  • ScoreA numeric or categorical evaluation attached to a trace.

Try it without code

Explore the demo

  1. Open the Langfuse public demo project from their docs.
  2. Click into a trace and find the slowest span and its cost.

Use it in code

Trace a RAG function with the @observe decorator.

python
# pip install langfuse ; set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
from langfuse import observe, get_client

@observe()
def retrieve(question):
    return ["Refunds take 5 days."]

@observe()
def answer(question):
    chunks = retrieve(question)
    reply = f"Based on our policy: {chunks[0]}"     # call your LLM here
    get_client().update_current_trace(user_id="u1", tags=["rag"])
    return reply

print(answer("How long do refunds take?"))
get_client().flush()
PRACTICE · TICK IT AFTER THIS

Instrument your RAG app with Langfuse, run your golden set, and attach judge scores to each trace.

Common mistakes

  • Forgetting to flush in short scripts, so traces never arrive.
  • Sending secrets in trace inputs.

Go deeper