When Tools Fail: Errors the Agent Can Recover From

Scenario

Example app: a sales assistant agent calls a get_quote(product_id, quantity) tool backed by a pricing service that times out a few times an hour.

What you see in production

Two failure modes alternate. Sometimes the user sees "Something went wrong" because the timeout exception escaped the agent loop. After a developer wraps the tool in try/except and returns an empty string, users instead get confident quotes with made-up prices: the model received nothing and filled the gap.

The naive code

def run_tool(block):
    try:
        result = TOOLS[block.name](**block.input)
    except Exception:
        result = ""                          # "don't crash the conversation"
    return {"type": "tool_result", "tool_use_id": block.id, "content": str(result)}

Reproduce it

A flaky pricing service, handled three ways:

class Timeout(Exception):
    pass

calls = {"n": 0}

def pricing_service(product_id, quantity):
    calls["n"] += 1
    if calls["n"] % 2 == 1:                     # every other call times out
        raise Timeout("pricing service timed out after 2s")
    return {"product_id": product_id, "unit_price": 12.5, "total": 12.5 * quantity}

def crash(args):
    return pricing_service(**args)

def swallow(args):
    try:
        return pricing_service(**args)
    except Exception:
        return ""

def structured(args, attempts=2):
    for attempt in range(1, attempts + 1):
        try:
            return {"content": pricing_service(**args)}
        except Timeout as e:
            last = e                            # transient: retry here, not via the model
    return {"is_error": True, "content": f"Pricing is unavailable right now ({last}). "
                                          "Do not estimate a price; tell the user you'll follow up."}

args = {"product_id": "P-42", "quantity": 10}
try:
    crash(args)
except Timeout as e:
    print("crash   ->", type(e).__name__, e)
calls["n"] = 0
print("swallow ->", repr(swallow(args)))
calls["n"] = 0
print("structured ->", structured(args))
calls["n"] = 0
print("structured, retries exhausted ->", structured(args, attempts=1))

crash   -> Timeout pricing service timed out after 2s
swallow -> ''
structured -> {'content': {'product_id': 'P-42', 'unit_price': 12.5, 'total': 125.0}}
structured, retries exhausted -> {'is_error': True, 'content': "Pricing is unavailable right now (pricing service timed out after 2s). Do not estimate a price; tell the user you'll follow up."}

Root cause

Tool failures are normal events in an agent, but neither naive version tells the model what happened. An escaped exception ends the conversation. An empty string looks like a successful call with no data, and a model asked for a price will try to be helpful. Retrying transient errors through the model is also wasteful: each retry costs a full model call.

The fix

Classify errors inside the tool, retry transient ones there, and return everything else as an explicit error result:

def run_tool(block):
    try:
        content = TOOLS[block.name](**block.input)            # tools retry timeouts internally with backoff
        return {"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(content)}
    except ValidationError as e:
        message = f"Invalid input: {e}. Fix the arguments and try again."
    except UpstreamUnavailable as e:
        message = f"{e.service} is unavailable. Do not guess its data; tell the user and offer a follow-up."
    except Exception:
        logger.exception("tool %s failed", block.name)
        message = "Internal error in this tool. Do not retry; apologise and offer a human handoff."
    return {"type": "tool_result", "tool_use_id": block.id, "content": message, "is_error": True}

Add an instruction in the system prompt: never invent values that a tool failed to return.

Trade-offs

• In-tool retries add latency; cap attempts and total time so the user is not kept waiting.
• Detailed error messages help the model but must not leak internals (stack traces, hostnames, secrets) into the conversation.
• Some errors deserve a hard stop (authorisation failures, policy violations); return them as final, non-retryable errors.

Detect it in production

• Error rate per tool and per error class; alarm on spikes.
• Sample answers given right after an error result and check they do not contain invented data.
• Count conversations that ended with an unhandled exception (should be zero).

Key points

• Return tool failures as explicit error results the model can read.
• Retry transient errors inside the tool, not through extra model calls.
• Tell the model never to invent data a tool failed to provide.

Further reading: Handling tool errors (is_error) · Retries & timeouts in LLM apps