Area 04 · Evals · Concept 17 of 20

OpenTelemetry for LLMs

The open tracing standard, with conventions for LLM calls, so traces work across tools.

11–20 · Production levelPhase 4 · Weeks 15–17≈ 1 hour

The idea

OpenTelemetry (OTel) is the vendor-neutral standard for traces, metrics and logs. Its GenAI semantic conventions define standard attribute names for LLM calls: gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens and so on.

If you emit OTel spans, you can send them to any compatible backend (Langfuse, Grafana Tempo, Honeycomb, Datadog) and switch later without re-instrumenting. LLM spans sit in the same trace as your HTTP, database and queue spans.

Instrumentation libraries (for example OpenLLMetry or OpenInference) wrap popular SDKs and emit these spans automatically.

Why it matters

It joins AI observability with the rest of your system’s observability and avoids lock-in.

Words to know

  • Semantic conventionsAgreed attribute names for common operations.
  • ExporterSends spans to a backend, often over OTLP.

Try it without code

Find the conventions

  1. Open the OTel GenAI conventions page.
  2. List the attributes you would set for one chat call.

Use it in code

Emit a GenAI span manually with the OTel SDK.

python
# pip install opentelemetry-sdk
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("rag-app")

with tracer.start_as_current_span("chat claude-sonnet-5-5") as s:
    s.set_attribute("gen_ai.operation.name", "chat")
    s.set_attribute("gen_ai.request.model", "claude-sonnet-5-5")
    # ... make the call ...
    s.set_attribute("gen_ai.usage.input_tokens", 1900)
    s.set_attribute("gen_ai.usage.output_tokens", 210)
PRACTICE · TICK IT AFTER THIS

Export your app’s spans over OTLP to a local Jaeger or Langfuse instance and view one full request.

Common mistakes

  • Custom attribute names that no tool understands.
  • Recording full prompts in spans that go to a shared backend.

Go deeper