Tutorials · 2026-09-08 · 1,477 words · 7 min read

An agent tool-use loop from scratch: states and stop rules

A tool-use loop built as an explicit state machine with six stop conditions and a token budget, measured over 2,000 simulated episodes. Code included.

agents tool use python budgets

An agent loop is about fifteen lines of control flow and about two hundred lines of everything that stops it. The fifteen lines are in every vendor tutorial. This article is about the rest: the state machine the loop actually walks, the six conditions that can end an episode, and the budget that decides which of them fires first. The listing is code/agent-tool-use-loop.py, it runs offline on the standard library, and the numbers below come from a real run of it, not an estimate.

The wire contract

Two vendors, one shape. Anthropic's documentation describes the loop as a while keyed on the stop reason: "Claude responds with stop_reason: "tool_use" and one or more tool_use blocks", your code executes them, and you send back tool_result blocks in a user message. The page states the exit condition plainly: "The loop exits on any other stop reason ("end_turn", "max_tokens", "stop_sequence", or "refusal")". OpenAI's function-calling guide has the same round trip with different nouns: the response carries output items of type function_call with a call_id, a name, and JSON-encoded arguments, and you append results as function_call_output objects carrying the matching call_id. Its advice on arity is worth copying into any client: "Since model responses can include zero, one, or multiple calls, it is best practice to assume there are several."

So the transport differs and the control flow does not. Everything below is written against an abstract turn that is either a list of calls or a final text answer, which is why the listing has no vendor SDK in it. Swapping in a real client means replacing one function.

The state machine

The loop in the listing is a machine with one live state and six terminal ones. The live state carries a step counter, running prompt and completion token totals, a count of consecutive error turns, and a counter keyed on the exact (tool name, canonical arguments) pair of every call made so far. That last counter is the only piece of state most tutorial loops omit, and it is the one that catches the failure mode that costs the most money.

check_stop runs before every request, not after, so a budget that is already exhausted never buys another round trip:

def check_stop(state: State, budget: Budget) -> str | None:
    """Return a terminal state name, or None to keep looping."""
    if state.step >= budget.max_steps:
        return STEP_CAP
    if time.monotonic() - state.started > budget.max_seconds:
        return WALL_CLOCK
    if not budget.enforce_guards:
        return None
    if state.tokens >= budget.max_tokens:
        return TOKEN_BUDGET
    if state.consecutive_errors >= budget.max_consecutive_errors:
        return ERROR_CAP
    if state.call_counts and max(state.call_counts.values()) > budget.max_repeats:
        return NO_PROGRESS
    return None

The six terminals are: the model answered; the step cap; the token budget; the wall clock; too many consecutive tool failures; and no progress, meaning the same call with the same arguments has now been issued more times than the budget allows. Every one of them returns a named reason rather than a bare None, because the reason is what you log, alert on, and later count.

Note what is missing. There is no terminal state for a tool raising an exception. In this design a tool never raises into the loop; the registry converts every failure into a result with an error flag, which matches what the API expects. Anthropic's page on handling tool calls documents is_error as an optional field on a tool_result and asks for useful text alongside it: "Write instructive error messages. Instead of generic errors like "failed", include what went wrong and what Claude should try next". The registry in the listing does that in its two failure paths, naming the available tools when the tool name is unknown and the required keys when an argument is missing.

Six stop conditions and the ones the API adds

The six above are the ones your loop owns. The API adds its own, and a loop that only branches on "is this a tool call" will misread them. The stop-reason reference lists seven values, including pause_turn for the case where "A server-tool loop reached its iteration limit" (the default limit is documented as 10 iterations per request), model_context_window_exceeded, and refusal. Two of those need handling that is not "stop": a pause_turn response should be appended and resent so the model can continue, and a max_tokens truncation in the middle of a tool call is not a stop at all. The same reference is explicit about it: "If Claude's response is cut off because it hit the max_tokens limit, and the truncated response contains an incomplete tool use block, you'll need to retry the request with a higher max_tokens value to get the full tool use."

What the budget costs and what it saves

The listing runs the same 2,000 seeded episodes twice, once with only the step cap and wall clock, once with the full guard set. Both arms see identical model draws, so the comparison is paired. The simulated model needs three distinct successful calls to answer and, on each turn, repeats its last call with probability 0.12, omits a required argument with probability 0.10, calls a tool that cannot help with probability 0.08, and with probability 0.10 the whole episode latches into repeating one call forever. Those four rates are inputs I chose, not measurements; what is measured is how the loop behaves under them. With seed 11 and a budget of 12 steps, 10,000 tokens, four consecutive errors and three repeats:

The guards cut mean cost per episode by 24.6% and cost 2.7 percentage points of completion. The interesting line is the breakdown of the 239 episodes the guards ended early: 185 were genuinely latched, and 185 is exactly the number of episodes that burned the full step cap in the other arm. The guards caught every stuck episode. The other 54, or 2.7% of all episodes, were healthy runs killed by a detector that fired too early. That is the trade in one sentence: a no-progress detector is a classifier, and its false-positive rate is a number you should know rather than a risk you accept blindly.

Where to set the detector

Tightening it is not free, and the shape of the cost is not linear. Sweeping only max_repeats and holding everything else:

Going from 2 to 3 buys 6.2 points of completion for 383 tokens an episode. Going from 4 to 6 buys nothing at all and costs 185. The knee is at 3 or 4 because a well-behaved model here retries a failed call once or twice and then changes approach, which is consistent with what the vendor documentation describes: "If a tool request is invalid or missing parameters, Claude will retry 2-3 times with corrections before apologizing to the user." Set the threshold above the model's own retry habit and below infinity, and measure where that is for your tools rather than copying my number.

Formatting rules that quietly break the loop

Three rules are easy to get wrong and none of them produce an obvious error. First, all results for one assistant turn go back in a single user message. The parallel tool use page says to "return one tool_result for each tool_use block, all together in the next user message", and warns that a separate user message per result "teaches" the model to stop calling tools in parallel, which shows up later as a latency regression nobody can explain. Second, within that message the result blocks must come first: "In the user message containing tool results, the tool_result blocks must come FIRST in the content array. Any text must come AFTER all tool results." Third, if you decided not to run one of the calls, you still owe a result for it, flagged as an error with a short reason.

One security note that belongs in the loop rather than the prompt. Tool results are the untrusted half of the transcript, and the same documentation says so: "Treat that content as untrusted: an attacker who can influence it may embed instructions that try to redirect Claude". Keeping tool output inside result blocks, never promoting it into the system prompt, is a structural defence that costs nothing to implement.

What this listing does not do

It does not stream, retry HTTP failures, or run calls concurrently, and its token accounting is a fixed cost per block rather than a real tokenizer, so the absolute token figures are the loop's arithmetic and not any model's billing. The state machine, the six terminals, the paired experiment and the sweep are the parts worth lifting. The _README for the dataset: arm is which guard set ran, episode is the seed index shared between arms, terminal is the terminal state, stuck marks the episodes that latched by construction, and steps, tokens, tool_calls and tool_errors are per-episode totals.