Tracing Agents: Answering "Why Did It Do That?"

Scenario

Example company: a customer complains that the support agent refunded the wrong order. The team has the final conversation transcript and the application logs.

What you see in production

The transcript shows the agent's final message; the logs show "tool call ok" lines. Nobody can see which order IDs the agent passed to which tool, what the tools returned, or why a short conversation cost almost 117,000 tokens. Reproducing it fails because the model does not behave identically twice.

The naive code

logger.info("agent started")
response = run_agent(messages)
logger.info("agent finished")                    # nothing about the steps in between

Reproduce it

A small tracer that records one span per model call and tool call, then explains the run:

import itertools
ids = itertools.count(1)
spans = []

def span(kind, name, parent=None, **attrs):
    s = {"id": next(ids), "parent": parent, "kind": kind, "name": name, **attrs}
    spans.append(s)
    return s["id"]

root = span("agent", "support_conversation", user="u-58")
span("llm", "plan", root, input_tokens=3_200, output_tokens=180)
span("tool", "search_orders", root, args={"email": "[email protected]"}, result=["A-1001", "A-1002"])
span("llm", "choose_order", root, input_tokens=61_000, output_tokens=120, note="full order history pasted into context")
span("tool", "refund_order", root, args={"order_id": "A-1001"}, result="refunded 59.00")
span("llm", "reply", root, input_tokens=52_000, output_tokens=90)

total = sum(s.get("input_tokens", 0) + s.get("output_tokens", 0) for s in spans)
print("total tokens:", f"{total:,}")
biggest = max((s for s in spans if s["kind"] == "llm"), key=lambda s: s["input_tokens"])
print("most expensive step:", biggest["name"], "-", biggest["note"] if "note" in biggest else "")
for s in spans:
    if s["kind"] == "tool":
        print(f"tool {s['name']}({s['args']}) -> {s['result']}")

total tokens: 116,590
most expensive step: choose_order - full order history pasted into context
tool search_orders({'email': '[email protected]'}) -> ['A-1001', 'A-1002']
tool refund_order({'order_id': 'A-1001'}) -> refunded 59.00

Root cause

An agent's behaviour is the sequence of model decisions and tool results, and none of it is in ordinary request logs. Because model output varies between runs, you cannot rely on reproducing a problem later; the evidence must be captured when it happens.

The fix

Trace every run as a tree of spans:

• Agent span for the task: user, task type, final status, total cost.
• Model spans: model, input and output tokens, latency, stop reason; optionally the prompt and response (subject to privacy rules).
• Tool spans: tool name, arguments, result summary, error flag, latency.
• Follow the OpenTelemetry semantic conventions for generative AI so traces work with standard backends.
• Store enough to replay a run offline: the model inputs and tool results let you re-run later steps with a changed prompt.

Trade-offs

• Prompts and tool results can contain personal data; redact, sample, or restrict access to full payloads, and set retention limits.
• Full payload capture is large; keep token counts and summaries for every run and full content for a sample or for flagged runs.
• Tracing adds a little latency and cost; batch exports asynchronously.

Detect it in production

• Dashboards for cost, steps and latency per task type, from the traces.
• Saved searches: runs with tool errors, runs that hit limits, runs where a destructive tool was called.
• A link from every support ticket to the trace of the conversation it refers to.

Key points

• Trace agents as a tree of model and tool spans with tokens, latency and arguments.
• Capture evidence at run time; agent runs are not reproducible later.
• Use OpenTelemetry's generative AI conventions and handle personal data deliberately.

Further reading: OpenTelemetry semantic conventions for generative AI · Observability for LLM features