The Agent Loop and the Runaway Bill

Scenario

Example app: a support agent answers order questions. It has one tool, search_orders(query). The loop is the textbook one: call the model, run any tool it asks for, send the result back, repeat until the model stops asking for tools.

What you see in production

Most conversations take 2 to 4 model calls. Then the bill for one day jumps: a single conversation made 180 model calls. The customer had mistyped their order number, search_orders kept returning nothing, and the model kept trying small variations of the same query until the request finally timed out.

The naive code

def run_agent(client, messages, tools):
    while True:                                        # stops only when the model decides to
        response = client.messages.create(model=MODEL, max_tokens=1024, tools=tools, messages=messages)
        messages.append({"role": "assistant", "content": response.content})
        if response.stop_reason != "tool_use":
            return response
        results = [run_tool(block) for block in response.content if block.type == "tool_use"]
        messages.append({"role": "user", "content": results})

Reproduce it

A scripted "model" that keeps retrying an empty search, run with and without a budget:

def fake_model(history):
    # Keeps rephrasing the same lookup while results are empty, like a model with no better idea.
    tries = sum(1 for h in history if h[0] == "tool")
    return ("tool", "search_orders", {"query": "order 4471" + "?" * (tries % 2)})

def search_orders(query):
    return []                                     # the order number was mistyped: nothing ever matches

def naive(limit_for_demo=180):
    history, calls = [], 0
    while calls < limit_for_demo:                 # stands in for "until the request times out"
        calls += 1
        kind, name, args = fake_model(history)
        history.append(("tool", name, args, search_orders(**args)))
    return calls

def bounded(max_steps=8, max_repeats=2):
    history, seen = [], {}
    for step in range(1, max_steps + 1):
        kind, name, args = fake_model(history)
        key = (name, args["query"].rstrip("?"))   # normalise trivial variations
        seen[key] = seen.get(key, 0) + 1
        if seen[key] > max_repeats:
            return step, "stopped: same search repeated, asking the customer to confirm the order number"
        history.append(("tool", name, args, search_orders(**args)))
    return max_steps, "stopped: step budget used up"

print("naive model calls:", naive())
print("bounded:", bounded())

naive model calls: 180
bounded: (3, 'stopped: same search repeated, asking the customer to confirm the order number')

Root cause

The naive loop hands the stopping decision entirely to the model. A model with no useful next action does not reliably give up: it keeps trying, because nothing in its context says that trying again is pointless. Nothing in the code caps steps, cost or repetition, so one stuck conversation can consume a day's budget.

The fix

Bound the loop in code and give the model information that lets it stop on its own:

def run_agent(client, messages, tools, max_steps=8, max_repeats=2):
    seen = {}
    for step in range(max_steps):
        response = client.messages.create(model=MODEL, max_tokens=1024, tools=tools, messages=messages)
        messages.append({"role": "assistant", "content": response.content})
        if response.stop_reason != "tool_use":
            return response
        results = []
        for block in (b for b in response.content if b.type == "tool_use"):
            key = (block.name, json.dumps(block.input, sort_keys=True))
            seen[key] = seen.get(key, 0) + 1
            if seen[key] > max_repeats:
                results.append(tool_error(block, "Already tried with these inputs. Ask the user for different details."))
            else:
                results.append(run_tool(block))
        messages.append({"role": "user", "content": results})
    return finish_with_handoff(messages)        # a polite final answer, never a silent timeout

Also make empty results informative: "No orders match 'order 4471'. Order numbers have 8 digits, for example 10004471." tells the model what to do next.

Trade-offs

• A low step limit cuts off legitimately long tasks; set it per task type (a research task needs more steps than an order lookup) and log every time it is hit.
• Repeat detection on exact inputs misses near-duplicates; normalise inputs or compare per tool.
• Stopping early is only acceptable with a good ending: a summary of what was tried and a next step for the user or a human.

Detect it in production

• Histogram of model calls per conversation; alarm on the 99th percentile and on any conversation above the step limit.
• Cost per conversation as a metric, with a daily budget alarm.
• Count of repeated identical tool calls per conversation.

Key points

• The agent loop needs a stop condition in code, not only in the model.
• Budget steps, repeats and cost per task type, and end gracefully when a budget runs out.
• Tool results should tell the model what to try next, especially when they are empty.

The basic tool-calling loop is introduced in Tool Calling & Agent Loops. Further reading: Tool use with Claude · Building effective agents