Trace LLM calls with OpenTelemetry

Approach

Wrap each model call in a span with OpenTelemetry's generative AI attributes (system, model, token usage) and record errors. Child spans for tool calls make agent runs readable as one trace. Avoid recording full prompts unless redacted and sampled.

Solution

import anthropic
from opentelemetry import trace

tracer = trace.get_tracer("support-assistant")
client = anthropic.Anthropic()


def traced_create(operation: str, **params):
    with tracer.start_as_current_span(f"chat {params['model']}") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.system", "anthropic")
        span.set_attribute("gen_ai.request.model", params["model"])
        span.set_attribute("gen_ai.request.max_tokens", params.get("max_tokens", 0))
        span.set_attribute("app.operation", operation)
        try:
            response = client.messages.create(**params)
        except anthropic.APIError as exc:
            span.record_exception(exc)
            span.set_status(trace.Status(trace.StatusCode.ERROR))
            raise
        span.set_attribute("gen_ai.response.model", response.model)
        span.set_attribute("gen_ai.response.finish_reasons", [response.stop_reason or ""])
        span.set_attribute("gen_ai.usage.input_tokens", response.usage.input_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", response.usage.output_tokens)
        return response


def traced_tool(name: str, func, **args):
    with tracer.start_as_current_span(f"execute_tool {name}") as span:
        span.set_attribute("gen_ai.tool.name", name)
        return func(**args)

Pitfalls

• Recording raw prompts and answers with personal data in traces; redact or omit.
• One span per conversation with no child spans, which hides where time went.
• Attribute names that differ per team; follow the semantic conventions.

Variations

• Use an auto-instrumentation package for the SDK if your tracing vendor provides one.
• Add the prompt version and route as attributes for comparisons.

Further reading: OpenTelemetry semantic conventions for generative AI