Choosing an Orchestration Pattern
A decision framework: task decomposability, verification cost, latency budget, and when a single agent with good tools beats a team.
Workflows vs. agents: the vocabulary
Anthropic's definitions, worth memorizing because interviewers use them:
- A workflow orchestrates LLM calls and tools through predefined code paths — you wrote the flowchart; the model fills in boxes.
- An agent dynamically directs its own process — the model decides which steps to take and which tools to call.
The prime directive: find the simplest solution possible, and only add complexity when it demonstrably improves outcomes. Many "agent" problems are actually one well-optimized LLM call with retrieval and a few good examples. Every layer of orchestration you add costs latency, tokens, and debuggability — it must pay for itself in measured results.
The pattern menu
Six composable patterns, each matching a task shape:
| Pattern | Shape of the task | Example |
|---|---|---|
| Prompt chaining | fixed steps known in advance; trade latency for accuracy | draft copy → check it → translate it |
| Routing | distinct input categories handled differently | refunds vs. tech support; easy queries → small cheap model, hard ones → big model |
| Parallelization: sectioning | independent subtasks on the same input | one call answers the user while another screens for policy violations |
| Parallelization: voting | same task run several times; confidence matters | 3 reviewers hunt vulnerabilities, flag on any hit |
| Orchestrator-workers | subtasks unknowable until the input arrives | which files change depends on the bug; which searches to run depends on the question |
| Evaluator-optimizer | clear criteria + iteration measurably helps | translation with a critic loop; draft → grade → refine |
Past all of these sits the autonomous agent: a loop of plan → act via tool → read environment feedback → repeat, with stopping conditions. Reserve it for open-ended problems where the step count is genuinely unpredictable.
The four questions that pick the pattern
1. Can you decompose the task in advance? Yes, into a fixed sequence → chaining. Yes, into categories → routing. Yes, into independent pieces → parallelization. No — decomposition depends on the input → orchestrator-workers. Not decomposable at all, just open-ended → agent.
2. How cheap is verification? Iterative patterns (evaluator-optimizer, autonomous agents) are only as good as their feedback signal. A compiler or test suite makes verification nearly free — agents thrive. If verification means "a human reads it carefully," iteration loops burn money without converging. Cheap verification is the single strongest argument for agentic patterns; expensive verification is the strongest argument against.
3. What's the latency budget? Chaining multiplies latency by step count. A user waiting on a chat reply won't tolerate a five-stage pipeline; a nightly batch job doesn't care. Parallelization is the exception — it spends tokens to buy time, cutting wall-clock latency by fanning out.
4. What's the task worth? Token economics from the research system: single agents ≈ 4x chat tokens, multi-agent ≈ 15x. A 15x-cost architecture needs a task whose value clears that bar — and that parallelizes cleanly. Most coding tasks fail the second test: subtasks share tight context (the same files and types), so parallel agents duplicate work or collide.
When a single agent beats a team
This is the trap the pattern menu invites: reaching for orchestration when the real problem is a weak single agent. Prefer one agent with good tools when:
- subtasks share context heavily — shipping the same background to five agents costs 5x and they still drift apart,
- the work is sequential by nature — each step depends on the last, so parallelism buys nothing,
- verification is the bottleneck — more generators just means more unverified output,
- your single agent fails for fixable reasons — bad tool descriptions, missing context, no feedback signal. Fix the agent-computer interface first. During Anthropic's SWE-bench work, the team spent more time optimizing tools than the overall prompt; requiring absolute file paths instead of relative ones eliminated an entire error class.
Compose, then simplify
The patterns are building blocks, not rival religions. A code-review product might route "style question" vs. "bug hunt," fan bug hunts out to parallel voting reviewers, and pass findings through one evaluator-optimizer round before reporting. That's three patterns in one pipeline — and each one should exist only because removing it measurably hurts results.
The interview-ready summary: decompose if you can, verify cheaply if you can't, parallelize only what's independent, and escalate to multi-agent only when the task is worth ~15x tokens and splits cleanly. Success isn't building the most sophisticated system — it's building the right one.