When to Use Multi-Agent Orchestration

Use multi-agent orchestration when one agent cannot reliably handle the job because domains, permissions, context, or parallel work need real separation. It is not the default upgrade from a chatbot. Start with one agent and tools. Add a router, supervisor, handoff, or parallel workers only when a measured failure maps to one of those patterns.

That advice is less exciting than drawing a swarm. It is also easier to operate.

What counts as multi-agent orchestration?

Multi-agent orchestration coordinates two or more specialized agents and decides what each one sees, does, and returns. The useful distinction is control, not agent count.

A router classifies a request and sends it to the right specialist.
A supervisor keeps control, delegates bounded tasks, and combines the results.
A handoff transfers control of a conversation or branch to a specialist.
Parallel workers explore independent parts of the same job, then return evidence to an aggregator.
A fixed workflow runs known stages in code. It may call several models, but the route is deterministic.

Microsoft’s orchestration guide puts direct calls, one agent with tools, and multi-agent systems on one complexity ladder. Anthropic draws a related line between fixed workflows and agents that choose their own process. Both recommend the lowest level of complexity that works.

The five-test decision matrix

The table below is our decision tool. Score each pressure from 0 to 2 using evidence from failed runs, traces, access reviews, or latency tests. Do not score from architecture taste.

Pressure	0	1	2
Domain separation	One domain and vocabulary	A few distinct request types	Specialists need different prompts, tools, or owners
Permission separation	Same access for the whole job	Some tool restrictions	Separate identities or data boundaries are required
Context pressure	One prompt stays focused	Context needs filtering	Domains need large, conflicting, or private context
Parallel value	Work is sequential	Some independent checks	Several independent searches or reviews dominate elapsed time
Ownership change	One agent should answer	A branch needs different behavior	A specialist must own the next response or decision

Use the total as a warning, not an automatic architecture generator:

0-2: use one model call or one agent with tools.
3-4: add deterministic routing, a fixed workflow, or on-demand skills before adding agents.
5-7: a router or supervisor may be justified. Prototype against the failure that raised the score.
8-10: multiple agents are plausible, but still test a simpler baseline.

The total is original analysis, not an industry standard. Its job is to make the reason for complexity falsifiable. A score without run evidence is decoration.

Which orchestration pattern fits the failure?

Observed failure	Pattern to test	Why it fits	Do not use it when
Requests fall into stable, distinct categories	Router	One classification step selects a specialist	Categories overlap so much that classification is unreliable
Several specialists help, but one agent must own the answer	Supervisor	The supervisor controls context, delegation, and synthesis	Every step and route is already known in advance
A specialist should take over a branch	Handoff	Control and conversation state move together	The specialist only supplies a fact to the current owner
Independent searches or reviews are too slow in sequence	Parallel workers	Separate context windows explore at the same time	Tasks share mutable state or depend heavily on each other’s output
Stages are known and ordered	Fixed workflow	Code controls the route and makes it testable	The next step depends on open-ended judgment

OpenAI’s orchestration documentation makes the ownership choice explicit: use handoffs when a specialist should take over, and agents-as-tools when a manager should stay in control. LangChain’s multi-agent guide frames the same design around context engineering: who sees which information, and when.

Use a router for clean categories

A router works when classification is easier than solving the request. Billing, technical support, and account recovery are separate enough to route. Ten product teams with overlapping responsibilities may not be.

The LangChain router pattern can send work to one specialist or fan out to several. Test the classifier separately. A router with a 90% routing score can still be unacceptable if the missed 10% contains account deletion or payment disputes.

Use a supervisor when one owner must stay accountable

A supervisor fits work where one agent needs the full user relationship while specialists stay behind the scenes. The supervisor chooses subagents, limits the context they receive, and checks their results before replying.

LangChain’s subagent pattern uses stateless workers for context isolation. That can prevent the main conversation from filling with domain instructions. It also creates a handoff contract you must test: task scope, source evidence, unresolved uncertainty, and the shape of the return.

Use handoffs when ownership really changes

A handoff is not a fancy tool call. It means a different specialist owns the next interaction. That is useful when a sales conversation becomes a security review, or a general support flow reaches an account-recovery branch with different controls.

Name the transfer in your traces. Track which agent owned the response, what state moved, and what state stayed private. If ownership is invisible, failures become arguments between logs.

Use parallel workers for breadth, not dependency

Parallel workers are strongest when several paths can be explored independently. Research, document review, and candidate generation often fit. A database migration usually does not.

Anthropic’s research-system report says its lead-agent and subagent design beat a single agent by 90.2% on an internal breadth-first research evaluation. The same report says the multi-agent system used about 15 times the tokens of chat interactions. It also warns that tasks with shared context or many dependencies are a poor fit.

That trade is rational for a high-value investigation. It is absurd for changing a shipping address.

When should you keep one agent?

Keep one agent when better instructions, retrieval, tool descriptions, or deterministic code fix the problem. These are cheaper failure modes to remove.

Use one agent if:

the same identity and permissions cover the whole task;
the context fits without mixing unrelated domains;
one trace is easier to test than several contracts;
the work is mostly sequential;
the next step can be expressed as code.

OpenAI’s practical guide recommends agents for workflows where deterministic rules struggle with judgment, exceptions, or hard-to-maintain logic. That does not mean every judgment needs a team of agents. One bounded agent can still be the right answer.

How do you test the architecture?

Run the simplest baseline and the proposed pattern on the same evaluation set. Grade outcomes, not how impressive the trace looks.

Collect 30-50 real tasks, including ambiguous and unauthorized requests.
Label the expected owner, tools, permission boundary, and final state.
Run one agent with the smallest useful toolset.
Run the proposed orchestration with the same models and source data.
Compare verified success, wrong-owner replies, unauthorized access attempts, latency, and cost per verified outcome.
Keep the multi-agent version only if the target failure improves enough to pay for its extra calls and contracts.

This extends the site’s cost-per-outcome method: tokens and latency belong in the denominator only after the result passes its checks. Use the agent evaluation framework to separate final-state, transcript, security, and latency graders.

What usually breaks in production?

The router becomes an untested model. Teams evaluate specialists and assume the dispatcher is plumbing. It is a decision-maker. Give it its own confusion matrix and high-risk test set.

Context crosses the wrong boundary. A specialist receives the full conversation because it was convenient. Context isolation then exists on the diagram, not in the system.

Nobody owns the final state. Workers return plausible text, but no component verifies that the ticket, record, or permission actually changed.

Parallel work duplicates itself. Three agents search the same sources because the delegation contract described a topic, not distinct questions.

The architecture cannot explain its bill. More agents increase model calls, tool calls, retries, and logs. If you cannot tie that cost to a measured gain, return to the simpler baseline.