When Multi-Agent Is the Answer, and When It Is Overhead

Multi-agent architecture has become the default thing to propose, and it is genuinely the right answer for a particular shape of problem. It is also, quite often, a way of turning one system you can reason about into five you cannot.

The decision is worth making deliberately, because it is expensive to reverse in either direction.

What splitting actually buys you

Three things, and it is worth being precise about them because they are the only real justifications.

Different tools with different permissions. An agent that can read the customer database and an agent that can issue refunds should not be the same agent, because the blast radius of a mistake differs by orders of magnitude. Separation here is a security boundary, and it is the strongest argument on the list.

Genuinely different context. If one part of the task needs a large body of legal text in context and another needs recent transaction history, keeping them together means carrying both for every call — slower, more expensive, and worse, because irrelevant context measurably degrades reasoning.

Independent iteration. If two parts of a workflow are owned by different teams and change on different rhythms, a boundary between them is an organisational necessity rather than a technical preference.

Notice what is not on that list: making the system seem more sophisticated, mirroring your org chart, or following the shape of the framework documentation.

What splitting costs

The costs are real and they compound.

Every boundary is a place to lose information. When agent A hands to agent B, something is serialised. Whatever nuance was in A's working state and not in the handover payload is gone. The characteristic multi-agent bug is a system where each agent behaves correctly and the whole produces nonsense, because a qualification got dropped in transit two hops ago.

Debugging becomes archaeology. A single agent that gave a wrong answer leaves one trace to read. Five agents leave five traces plus the handovers between them, and "why did it say that" becomes a reconstruction exercise. Without tracing across the whole run this is close to impossible, and that tracing is work most teams do only after they need it.

Latency and cost multiply. Each hop is another model call, often with overlapping context. A five-agent pipeline is not five times more capable than one agent; it is reliably five times slower and more expensive.

Error rates compound. If each step is 95% reliable, five sequential steps are about 77% reliable. That arithmetic is unforgiving and it is the single most underestimated fact in multi-agent design.

A test that works

Before adding an agent, ask whether the boundary would still exist if you were writing this as ordinary software with no models involved.

If the answer is yes — this is a separate service, with separate permissions, owned by a separate team, called over a network — then the boundary is real and modelling it as a separate agent is sensible. If the answer is no, and you are splitting only because the prompt got long, then you are converting a prompt-engineering problem into a distributed-systems problem, which is a considerably worse problem to have.

The long-prompt version usually has a better answer: better retrieval, so less has to be in context; or tools, so the agent fetches what it needs rather than carrying everything.

If you do split, structure it

Two patterns hold up in production, and one does not.

Supervisor with specialists works. One agent owns the plan and delegates bounded sub-tasks to agents with narrow scope and narrow permissions, each returning a structured result. Control is centralised, the trace is legible, and each specialist is independently testable.

A fixed pipeline works when the sequence genuinely is fixed. If the steps always happen in the same order, that is a workflow and it should be written as one — with model calls at the steps that need judgement. Considerably easier to test and to reason about than anything dynamic.

Free-form peer negotiation does not work reliably. Agents deciding among themselves who does what, passing messages until they converge, is the version that demonstrates beautifully and fails in ways nobody can reproduce. It is also the version that runs up a bill while looping.

Two things to build regardless

Whatever the topology: a single trace identifier that follows the whole run across every hop, so one query reconstructs what happened; and a hard ceiling on total steps and total spend per run, enforced outside the agents themselves.

Agents that can call agents can loop. Without an external ceiling, the first time that happens you will find out from the invoice.

The short version

Split for permissions, for genuinely disjoint context, or for team boundaries. Do not split for elegance. And whatever you build, make sure one person can sit down with one trace and explain what the system did — because the day that becomes impossible is the day the system becomes unmaintainable, regardless of how well it performs.