The 3am bill
In one line
A loop with no bound did not error; it worked all night, expensively, on a task it had stopped making progress on at step two.
The situation
The billing assistant answers "why was my invoice higher this month" by calling a few tools in a loop. It has been fine for weeks. At 03:10 a usage service starts returning empty arrays. Nobody is paged, because nothing errors. At 08:30 someone notices the overnight model spend is forty times normal.
The had been asking get_usage for the same period
with slightly different argument formats, several thousand times, across
hundreds of conversations, each one appending the empty result to its context
and trying again.
Why nothing fired#
Every alert the team had was built for failures that announce themselves: error rates, timeouts, 5xx responses. The agent produced none. Each step was a successful model call followed by a successful tool call that returned a valid, empty result. From the infrastructure's point of view, it was a busy night.
The loop has no natural bound. The model decides when it is done, and a model that has not found what it is looking for does not conclude that it never will. It tries a different date format. It tries again. It is, in a precise sense, working — just not usefully.
The four bounds#
Each is a few lines of code, and the incident needed any one of them.
A step cap. Eight iterations, then stop and report. Set at the lowest number real traffic needs, and alert when a run hits it, because hitting it almost always means a tool is misbehaving.
A spend cap. Tokens or dollars per run, whichever is easier to count. A that trips on budget rather than on error rate is the control this class of failure needs, and it is unusual only because the thing being tripped on is money rather than a 500.
Repeat detection. The same with the same arguments twice in one run is a loop, not a strategy. Stop, or at least stop calling that tool.
Actionable empty results. [] gives the model nothing to do except try
again. {"error": "no_usage_for_period", "available": ["2026-07", "2026-09"]}
gives it a way forward or a reason to stop. This is a change to the tool, not
to the agent, and it is often the one that turns the loop into a correct
answer.
Then make the loop visible. One per iteration in the request's , with tokens and cost attached, so "step count per run" is a metric — and its p99 is the alert that fires at 03:15 instead of 08:30.
Questions to ask
- What is the maximum number of steps a run can take, and where is it enforced?
- What is the maximum spend per run, and what happens at the limit?
- What does a tool return when it has nothing — and does that tell the model what to do next?
- Is step count per run on a dashboard, with an alert on its tail?
Mindset
An agent is a while loop whose condition is a model's opinion. Give it every
bound a while loop normally has — a counter, a budget, a progress check —
and remember that the failure you are guarding against raises no exception.
Where it connects
Phase 4 · 7 min
Agents & tool use
A loop where the model picks a tool and your code decides whether to run it, and every interesting problem is in the loop rather than in the model.
Phase 4 · 6 min
Observability
Aggregate metrics tell you something regressed; only a trace of the whole chain tells you which of six stages did it.
Got the shape of it?
Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.