Workflow patterns

The steps are known and ordered. Each model call does one thing, and your code calls them in sequence.

extract fields  ->  classify  ->  draft reply  ->  validate

Everything good follows from the control flow being yours. Cost is predictable — four calls, every time. Latency is predictable. Each step has its own prompt, its own structured output schema, and its own Eval, so when quality drops you know which link moved. Failures are ordinary exceptions at ordinary boundaries.

A chain also lets you use different models per step, which is usually the largest cost win available. Extraction and classification run on a small cheap model; only the drafting step needs a large one.

Most problems people reach for agents to solve are chains. If you can write down the steps, write down the steps.

A cheap call decides which path to take, and each path is its own chain or its own model.

classify intent
  ├── billing    -> billing chain, small model
  ├── technical  -> RAG chain, large model
  └── other      -> escalate to a human

This is model routing, the highest-leverage pattern for cost, and it is underused. A small model handling 70% of traffic at a twentieth of the price, with the large model reserved for the cases that need it, changes the economics of a product without changing its quality — provided the router is accurate, which is a classification problem with a clean Eval.

Route on confidence too, not only on category: when the classifier is unsure, escalate rather than guessing. That single branch turns a quality problem into a capacity problem, which is a much better one to have.

Independent subtasks run concurrently and something merges the results. Two distinct uses:

Decomposition. Summarise twenty documents at once, then synthesise. Latency is one call instead of twenty, at the same token cost.

Voting. Run the same task three times and take the majority, or run three different prompts and compare. This buys real accuracy on high-stakes classification for a linear increase in cost, and it is the simplest reliability technique available.

The related shape is evaluator-optimiser: generate, critique with a second call, revise. It helps most where quality is judgeable but not computable — writing, code review, translation — and it is essentially LLM-as-judge wired into the production path rather than the test suite. Bound the number of revision rounds, because the second round rarely helps and the third never does.

The agent loop from the previous topic. The right shape when the steps genuinely cannot be known in advance — debugging an unfamiliar system, multi-hop research where each finding determines the next query.

It is the only pattern with unbounded cost, and the only one where a failure can be silent rather than exceptional. Everything about operating it is about imposing the bounds the pattern does not have: a step budget, a spend cap, a circuit breaker, repeat detection.

The honest heuristic: reach for a loop when you have tried a chain and the chain genuinely could not express the task. Not before.

Real systems combine all four. A production support assistant is typically a route at the top, chains on most branches, and a bounded loop on exactly one branch where investigation is needed.

The property to preserve is that the expensive, unpredictable shape is contained. An agent as one branch of a route, with a step budget, is a manageable component. An agent as the top-level architecture makes every request unpredictable, including the 70% that were simple.

Two more composition rules worth stating. Put the Guardrails at the boundaries between stages rather than inside prompts, so validation is ordinary code. And make every stage independently traceable — a Tracing span per stage is what lets you answer "which step regressed" without rerunning anything.

A support system handling 10,000 tickets a day. Version one is a single agent with six tools, every ticket.

avg model calls per ticket        7.2
avg tokens per ticket           24,000
p99 latency                       48 s
cost per day                    $1,440
resolution rate                    73%
debuggability                   one trace, six similar steps

Version two, restructured as a route over chains, with one agent branch:

route (small model, 1 call, 600 tokens)
  ├── 58%  password / access   -> 2-step chain, small model
  ├── 24%  billing question    -> RAG chain, large model
  ├── 12%  technical issue     -> bounded agent, max 6 steps
  └──  6%  unclear             -> human

avg model calls per ticket        2.1
avg tokens per ticket            6,800
p99 latency                       11 s
cost per day                      $390   (-73%)
resolution rate                    79%   (+6)
debuggability                   per-branch evals and dashboards

Cost fell by nearly three quarters and quality rose. The quality gain is the interesting part: narrow prompts on narrow branches outperform one prompt trying to cover everything, and the 6% escalation branch removed the cases where the agent previously produced a confident wrong answer.

Tail latency improved for a structural reason too — the bounded-agent branch is the only one that can run long, and it is now 12% of traffic instead of 100%.