Multi-Agent Orchestration: Safe Patterns for the Enterprise
A single agent that parses one document or summarizes one transcript is easy to run. A workflow with specialized reasoning, strict compliance and sequential handoffs is harder, and splitting it across several agents is a common answer. It is also an expensive one, and the evidence for it is thinner than the enthusiasm suggests. This guide covers when to split, which orchestration patterns hold up, how to brief and constrain each agent, how to keep state, and what stops one bad output from spreading.
Prove you need more than one agent
Anthropic's account of its multi-agent research system is the most candid public data point. A lead Claude Opus 4 agent with Claude Sonnet 4 subagents outperformed single-agent Opus 4 by 90.2% on its internal research evaluation. The same post says agents use about 4 times the tokens of a chat interaction and multi-agent systems about 15 times, and that domains where all agents need the same context, or that have many dependencies between agents, are not a good fit today. Cognition's Don't Build Multi-Agents argues the same from the failure side: share full agent traces and not just messages, because actions carry implicit decisions and conflicting decisions give bad results, so parallel subagents can contradict each other. Its default is a single-threaded linear agent. A broader failure analysis, Why Do Multi-Agent LLM Systems Fail?, catalogs 14 failure modes in three categories across 1,600-plus annotated traces from seven frameworks, and reports that gains on popular benchmarks are often minimal.
Use these as a gate. Split a workflow only when the parts have separable context, when different steps need different permissions, or when independent branches can run in parallel and be merged. Otherwise keep one agent with good tools and a short, checked prompt. Permission separation is the strongest of those reasons, and the one that does not depend on benchmark gains. A single agent that retrieves, analyzes and executes needs privileges across every connected system, while separate agents can each hold the minimum.
Orchestration patterns
In a hierarchical pattern, an orchestrator breaks the request into tasks, delegates to specialists and compiles results. It suits tasks with clear dependencies, such as a legal review where one agent extracts clauses, another checks them against policy and a third summarizes, and the orchestrator makes sure the compliance step never sees an unvalidated extraction. In a sequential pipeline each agent's output feeds the next. It is the simplest pattern and the easiest to audit, since the state between stages can be inspected when something fails. Networked, peer-to-peer collaboration lets agents talk directly. It can loop or converge on a wrong answer, so it needs a hard cap on interaction cycles and a deterministic fallback that halts the process. Choose the least flexible pattern that fits the task.
Brief each agent like a contractor
Specialization fails at the handoff. In the Anthropic post, early lead-agent instructions like "research the semiconductor shortage" were vague enough that subagents misread the task or repeated each other's searches: one explored a 2021 chip crisis while two others duplicated work on 2025 supply chains. The fix was a clear objective, an output format, guidance on tools and sources, and explicit task boundaries for each subagent. Write that brief as a schema and validate it. Define each specialist's input and output schemas, reject output that does not conform before it reaches the next agent, and treat free text between agents as a defect.
Routers, specialists and critics
A router chooses a destination and does no deep work. Make it a constrained choice among the options you actually have, and include an explicit "none of these" option. Framework defaults are worth checking at this point. In the OpenAI Agents SDK, the handoffs documentation says handoffs are represented to the model as tools (a refund specialist becomes transfer_to_refund_agent), so routing is ordinary tool selection and gets worse as the list grows. It also says the receiving agent, by default, "gets to see the entire previous conversation history". A specialist that should only see the part of the request it needs will see everything the router saw, including tool results and notes from earlier steps, unless you add an input filter. Pass the specialist a trimmed brief and treat the default history as a leak to close, not a convenience.
Specialists get narrow tools, context and prompts. A critic evaluates output against rules and can reject it for revision. Where a rule can be checked in code, such as a schema, a total that must balance or a clause that must appear, check it in code and keep the critic model for judgments code cannot make. Cap revision rounds, because a critic and a producer can trade the same objection indefinitely. Two is a reasonable starting point. When the cap is hit, release the output with a flag or escalate to a person. Cap tool-call iterations per run as well, so one odd request cannot run up model calls unnoticed.
State and persistence
A shared context object that every agent appends to is simple, and it grows until it overflows the window or buries the instruction that matters, so pass each agent only what its step needs. For long or complex workflows, keep state in a database. Agents read and write the specific records they need, the context stays small, and a crash resumes from stored state instead of restarting. Store the state between agents as typed records with a version, so a retry of one step cannot overwrite a newer result from another.
Failure handling
When an agent returns invalid output or times out, trigger a deterministic fallback: a fixed message, a simpler rule-based path or escalation to a person. Retry transient errors such as timeouts and rate limits with backoff, but do not resend the same prompt to an agent that keeps producing the wrong answer. Change the prompt or the model instead. Add a circuit breaker per agent so a high error rate disables that agent and routes to the fallback while the cause is fixed.
Security and traceability
Give each agent the minimum permissions for its task, which limits the damage if prompt injection takes one over. An agent that drafts email should not be able to send it, and a separate, restricted component should do the sending. Keep tool execution deterministic. The agent emits a structured command, and ordinary code in a sandbox or service runs it, so the model never executes anything itself. Scrub personal data before it reaches an external provider. Log every agent's input, tools and output under one trace ID so you can find which agent produced a bad result, and watch metrics such as response time, tool success rate and fallback frequency, since a jump in fallbacks often means an upstream API changed or a prompt update degraded one agent.
Example: a legal review workflow
A user uploads a contract. The orchestrator sends it to an extraction agent that returns key clauses (termination, liability limits, payment terms) as structured JSON. A deterministic validator checks the JSON against the schema, and only then does the compliance agent compare the clauses with company policy and report discrepancies. The summary agent receives the original document, the extracted data and the compliance report and writes a short summary of the key risks. The orchestrator tracks state, handles errors and does not advance until each stage's output validates. This one has real separation to justify it: extraction needs the document, compliance needs only structured data and policy, and the summary agent is the only one that sees both, so each holds the narrowest context and permission set.
Start with one multi-step process that is currently handled by a single agent or by hand. Split it only where the context or permissions separate, write each handoff as a schema, and put caps and fallbacks around every loop before adding the next agent.
References
- Anthropic, multi-agent research system engineering post for the token multipliers, the 90.2% result, poor-fit domains and the subagent briefing lessons.
- Cognition, Don't Build Multi-Agents for shared-context and implicit-decision principles and the single-threaded default.
- Why Do Multi-Agent LLM Systems Fail? for the 14-mode failure taxonomy over 1,600-plus annotated traces.
- OpenAI Agents SDK, Handoffs for handoffs as tools and the default full-history transfer with input filters.