A multi-agent system splits one job across several language-model agents that send each other messages. Each agent runs its own loop with its own context, and some rule combines what they produce into one answer. This section asks when that beats a single agent. It answers with a few small models you can solve exactly, and then with the measured evidence. It matters because a multi-agent design costs several times the tokens of one agent, and in a design review or an interview you will be asked to justify that cost with numbers.

Practitioners disagree sharply about it. In June 2025 two well-known teams published opposite advice a day apart. Anthropic described the system behind Claude's Research feature, in which a lead agent plans the work and launches helper agents that run in parallel. It reported that this system beat a single Claude Opus 4 agent by 90.2% on an internal research evaluation (Anthropic 2025). Cognition, the company behind the coding agent Devin, published a post titled "Don't Build Multi-Agents" (Yan 2025). And a Berkeley study of 1,642 runs of seven open-source multi-agent frameworks found that they failed on 41% to 86.7% of their tasks (Cemri et al. 2025).

All three are right, for different tasks. Several agents can buy three things that one agent cannot have. Their errors may be independent, so a vote can cancel them. They can work in parallel, which cuts the time to an answer. And each one starts with an empty context window. They also cost three things. Their errors are often not independent after all. Coordinating them takes time and tokens. And a decision one agent makes may never reach another.

We put a number on each of the six. The tools are a voting theorem, a model of shared errors, a model of debate, a formula for parallel speedup, and a count of the tokens that agents read from each other. Then we read the published results through those numbers and end with a rule for when several agents beat one.

What counts as a multi-agent system

Start with one agent. In §24.1 an agent is a language model in a loop. It reads its history so far, which is the task, its earlier thoughts and actions, and what the tools returned. Then it writes either an action, such as a tool call, or a final answer. In symbols it is a policy $\pi(a_t \mid h_t)$, a probability distribution over the next action $a_t$ given the history $h_t$. Every earlier section of this chapter studied one such agent with one growing context. The one exception was the protocol by which one company's agent hands a task to another's, in §24.6. A multi-agent system wires several agents together.

In plain words, a multi-agent system is a set of agent loops, a wiring diagram for their messages, and a rule for combining what they produce. Two systems with the same agents but different wiring are different systems. Most of this section is about the wiring.

Most designs in use follow one of five patterns. Two of them come from the workflow patterns of §24.1, fixed arrangements of model calls written in code (Anthropic 2024). Parallelization runs several model calls at once, either on different subtasks (called sectioning) or on the same task several times (called voting). Orchestrator-workers has a lead agent hand out subtasks.

Debate in this sense is a way for cooperating agents to get a question right. It differs from the debate for oversight of §19.5, where two debaters argue opposite sides and a weaker judge decides between them.

Two more patterns complete the list. In a pipeline with roles, agents with fixed jobs (product manager, architect, engineer, tester) pass structured documents down a fixed sequence. In a mixture of agents, several different models answer, and an aggregator model writes one answer from all of theirs, possibly over several layers. Here are the five side by side.

The table already says where the benefits and costs live. Voting works only if errors cancel. Orchestration works only if the pieces are separable. Debate pays for reading in a way that grows with the square of the number of agents. The rest of the section makes each of these precise.

Three voters, worked by hand

Before any theory, here is the first idea of the section in one calculation.

So more agents help to the extent that their errors are independent. The second benefit has nothing to do with voting. $n$ workers can do $n$ things at once, and each starts with an empty context window. That matters because a single agent's context fills with everything it has read, which is why §25.2 treats subagents as a tool for managing context. We make the statistical benefit precise first.

Voting among independent agents

Voting among agents is the same thing as self-consistency from §18.2. There you sampled several answers from one model and kept the most common one. That section showed that, as the number of samples grows, the vote settles on the answer the model gives most often. Here we want a rate, how fast the vote improves as agents are added.

The classical answer is the jury theorem, published by the Marquis de Condorcet in 1785 (Condorcet 1785). Its idea fits in one sentence. If each voter is right more often than a coin and the voters err independently, then a large enough majority is almost never wrong. With $p = 0.7$, one voter is right 70% of the time, three are right 78.4% of the time (the example above), and five are right 83.7% of the time.

To see how fast that climbs, we need one tool from probability. It bounds how far an average of independent coin flips can stray from its mean, and §15.1 used it to build the exploration bonus of the UCB bandit algorithm.

In plain words, the average of $n$ independent bounded variables rarely strays more than $u$ from its mean, and the chance that it does shrinks exponentially in $n u^2$. We use it as a known result (Hoeffding 1963). Its proof bounds the moment generating function of each variable and then applies Markov's inequality.

Now the jury theorem itself, in the form a two-answer vote among agents needs. Write $P_n(p)$ for the probability that the majority of $n$ independent voters is right when each is right with probability $p$.

In plain words, a crowd of independent voters, each a little better than a coin, is almost always right. Each pair of extra voters helps by an amount proportional to how much better than a coin they are, $2p - 1$. Part 2 is why experiments use odd numbers of agents, since the extra voter in an even jury only creates ties. Part 3 says the error falls exponentially in $n$.

The cell computes $P_n$ exactly and compares the true error with the bound of Part 3.

At $p = 0.7$ the majority is right with probability 0.700, 0.784, 0.837, 0.922, and 0.9986 for $n = 1, 3, 5, 11, 51$. Each even jury repeats the odd value below it, as Part 2 says. The bound is loose for small juries. At $n = 1$ it allows an error of 0.92 against the true 0.30. Its exponential rate is right, though. At $n = 51$ it promises an error below 0.017, and the truth is 0.0014.

In practice this tells you what to ask before adding voters. Is a single agent right more often than not, and are the agents' errors independent? An interviewer who asks "why does voting help?" wants those two conditions and the exponential rate. The next part shows that independence is the condition that fails in practice.

from math import comb

def majority_accuracy(n, p):
    raise NotImplementedError

def smallest_n(p, target, n_max=100001):
    raise NotImplementedError

Shared errors cap what voting can gain

The jury theorem rests on independence, and copies of one model violate it. They share weights and training data, so they share blind spots. To see what that costs, we need a model of correlated voters that is small enough to solve.

Here is the model in words. On some questions every copy behaves identically, because the difficulty lies in the model itself. On those, all copies are right together or wrong together. On the rest, the copies err independently. One number says what fraction of questions are of the first kind.

The symbol $\rho$ is a correlation. The correlated defense layers of §19.7 used one too, because safety filters trained on similar data fail together for the same reason copies of a model do. To check that $\rho$ is the correlation, let $X_i$ be 1 when agent $i$ is correct and 0 otherwise. For two agents, $\mathbb{P}(X_i = X_j = 1) = \rho p + (1-\rho)p^2$, so

In plain words, $\rho$ is exactly the correlation between two agents' correctness. You can estimate it on labeled data by counting how often two agents are wrong on the same questions.

In plain words, a shared blind spot puts a floor under the error that no number of copies can lift. With $\rho = 0.5$ and $p = 0.7$ the ceiling is $0.35 + 0.5 = 0.85$. The quantity $n_{\text{eff}}$, the effective number of voters, is the number of independent voters that would give the same spread. At $\rho = 0.5$, 101 voters are worth fewer than two independent ones ($n_{\text{eff}} = 1.98$).

This is the reason behind the advice to mix different models. Diversity helps because it lowers $\rho$, not because any one model is better.

Make two predictions before you run the next cell. With $p = 0.7$ and $\rho = 0.2$, what accuracy will 51 voters reach, compared with the independent 0.9986 and with the ceiling $0.2 \cdot 0.7 + 0.8$? And how many independent voters are 101 correlated voters at $\rho = 0.2$ worth? The cell checks the formula against a simulation. Each simulated question first flips a biased coin, heads with probability $\rho$, to decide whether the agents share one draw.

Simulation and formula agree to within a few standard errors of the simulation, and one standard error is about 0.002 at 40,000 trials. At $\rho = 0.2$, 51 voters reach 0.939 instead of 0.9986, against a ceiling of 0.94, and 101 voters are worth 4.8 independent ones. At $\rho = 0.5$ the curve is within 0.001 of its ceiling of 0.85 by $n = 51$.

Plurality votes with many answers

Most agent tasks have more than two possible answers. A math problem has one right number and many wrong ones, and the vote takes the plurality, the most common answer. Spreading the wrong answers over many options helps the vote a great deal. As §18.2 showed for self-consistency, the right answer only has to beat each wrong answer separately, not all of them combined.

Here is a small simulation. There are $K$ possible answers. Each agent is right with probability $p$, and otherwise it picks one of the $K - 1$ wrong answers uniformly at random.

With $p = 0.4$ and two answers, the vote gets worse with every agent, from 0.40 down to about 0.07 at $n = 51$, as the jury theorem predicts for $p \lt 1/2$. With four answers, each wrong one has probability 0.2, so the right answer is the most likely one, and 51 voters are right 93% of the time. With ten answers they are right essentially always.

What matters for a plurality is whether the right answer is the single most likely one. Under even spreading that means $p > (1-p)/(K-1)$. The shared-draw argument goes through unchanged. On a shared question every agent gives the same answer, so the vote is right with probability $p$. So the plurality's ceiling is $\rho p + (1-\rho)$ when the right answer is the most likely one, and $\rho p$ when some wrong answer is more likely.

The widget puts the two effects side by side. Move $\rho$ and watch the ceiling drop, then raise the number of answers $K$ and watch a vote that was hopeless at $p \lt 1/2$ start to work.

Real wrong answers are not spread evenly. A model's errors cluster on a few tempting mistakes, which acts like a high $\rho$ for a plurality vote. §18.2 showed the extreme case. When the model has a favorite wrong answer that is more likely than the right one, the vote converges to the wrong answer.

The same idea explains two results that look like advertisements for many agents. Li et al. (2024) found that sampling and voting with more agents raises accuracy steadily. On GSM8K, about 15 Llama2-13B agents matched one Llama2-70B, and the gains were larger on harder tasks and for weaker models. In this section's terms, a weak model on a hard task has many questions it gets right only somewhat more often than it gets them wrong. Those are the questions on which extra draws still change the vote.

Wang et al. (2024) built a mixture of agents with six different open models as proposers and one aggregator. They scored it on AlpacaEval 2.0, which asks a judge model whether an answer beats a reference answer, with a correction for answer length. The mixture reached a win rate of 65.1%, against 57.5% for GPT-4o alone. An ablation, a run with one ingredient removed, found that proposals from several different models consistently beat the same number of proposals sampled from one model. That is the effect of a lower $\rho$, measured.

In practice, measure $\rho$ before you pay for voters, and spend the budget on whatever lowers it. In an interview, expect "five copies of one model vote, and accuracy barely moves. Why?" The answer is the ceiling $\rho p + 1 - \rho$, an estimate of $\rho$ from data, and a fix that lowers it.

import numpy as np

def shock_accuracy(n, p, rho):
    return majority_accuracy(n, p)          # placeholder: assumes independent agents

def shock_ceiling(p, rho):
    raise NotImplementedError

Debate, and why it often only matches a vote

Voting treats the agents as experts who never talk. Debate lets them talk. Du et al. (2023) had three copies of ChatGPT answer a question. Then each copy read the others' answers and reasoning and revised its own. A majority vote over the final answers gave the result. Their headline table is the standard citation for multi-agent debate. In the self-reflection row, one agent critiques and revises its own answer.

Before believing that talking helped, count the generations. The vote used 3 answers per question. The table's caption says "two rounds of debate". In the paper's released code the first answers are the first of those rounds, and only the second reads the peers. So debate used 3 first answers plus 3 revisions, 6 generations in all, twice the compute of the vote. Each revision also read the other agents' outputs. The fair comparison is a vote over 6 independent answers, and the table does not include it.

Later work tested debate against such baselines. Smit et al. (2024) tested several debate protocols on seven datasets and tracked what each cost. Debate did not reliably beat self-consistency or ensembles of reasoning paths, and the best protocols were very sensitive to their settings. Tuning how strongly agents were told to agree did lift debate above the other methods. Choi et al. (2025) split debate into its vote and its talk. On seven benchmarks, a majority vote over the agents' first answers alone accounted for most of the gains attributed to debate, with no revisions at all. They also proved that, in a Bayesian model of agents that are alike and all read each other, talking leaves each agent's expected belief in the right answer unchanged from round to round.

A model small enough to solve exactly shows why. It isolates the one thing debate adds to a vote, agents changing their minds after reading each other. It is a Markov chain, a random process whose next state depends only on its current state. The state is the number of agents that are currently right.

A tiny case makes the rule concrete. Take $n = 3$ agents with $k = 1$ right. Each wrong agent sees one right peer out of two, so it switches with probability $\alpha/2$. The right agent sees two wrong peers out of two, so it switches with probability $\beta$.

The two numbers $\alpha$ and $\beta$ carry the real question. If a correct argument convinces more often than an incorrect one, $\alpha > \beta$. That is the hope behind debate. It is the same bet that debate for oversight makes, that refuting a lie is easier than defending one (§19.5). If agents just conform to their peers whatever the merits, $\alpha = \beta$. If they defer to confident wrong answers, $\beta > \alpha$.

Two terms are needed for the result. A state is absorbing if the chain never leaves it once it arrives. A process is a martingale if its expected next value, given everything so far, equals its current value, so on average it does not drift.

In plain words, talk among agents that only conform moves the group toward agreement, not toward the truth. Take $p > 1/2$ and $n \ge 3$. By the jury theorem, the first majority is right with probability $P_n(p) > p$. Conforming talk turns it into a consensus that is right with probability $p$, the accuracy of one agent. So with enough rounds, talking is worse than not talking at all. Debate helps only to the degree that right answers persuade more than wrong ones. Even then it must beat a vote that spends the same compute on more independent answers.

Liang et al. (2023) saw the single-agent version of conformity. A model asked to reflect on its own answer stopped producing new ideas once it was confident, even when it was wrong. They called this degeneration of thought. Their remedy was to make two debaters argue against each other in front of a judge, which keeps disagreement alive on purpose.

The cell builds the chain's transition matrix, the table of probabilities of moving from each count $k$ to each other count in one round. It runs the chain for four rounds and compares each round with a vote that spends the same number of generations on independent answers.

Read the rows with $n = 3$ agents at $p = 0.6$. Before any talk, the vote of three is right 64.8% of the time. With $\alpha = \beta = 0.5$ the accuracy falls round by round toward 0.6, as the theorem says. With $\alpha = 0.9$ and $\beta = 0.1$ it climbs to 0.829 after one round of revision, the setting of the Du et al. table. That is well above the 0.683 of a vote over six answers. After two rounds it reaches 0.890, against 0.733 for nine answers.

The widget lets you find that boundary yourself. Set $\alpha = \beta$ and watch the debate curve sink toward $p$ while the equal-compute vote climbs. Then raise $\alpha$ until debate pulls ahead.

In practice, never report a debate result without a vote over the same number of generations next to it. An interviewer who hears "debate raised GSM8K from 77 to 85" will ask for that baseline, and the reason it matters is the martingale.

import numpy as np

def debate_matrix(n, alpha, beta):
    return np.eye(n + 1)                     # placeholder: nobody changes their mind

def debate_accuracy(n, p, alpha, beta, R):
    raise NotImplementedError

Parallel work, and the limit on speedup

Now the second benefit, doing work at the same time. Suppose the job is not one question answered many times but one big task that splits, such as surveying a hundred companies or reading every caller of a function. Subagents can do the parallel part at once. How many should the orchestrator start?

The first answer is an argument about parallel computers (Amdahl 1967). Some of every job is serial, such as planning, combining results, and writing the report, and no number of workers speeds that part up. Measure time in units of one agent's time on the whole task. If a fraction $f$ of the work splits evenly among $n$ workers and the rest, $1 - f$, is serial, then $n$ workers take $(1 - f) + f/n$. With $f = 0.9$ and 10 workers the time is $0.1 + 0.09 = 0.19$, a speedup of 5.3. No number of workers can beat $1/(1-f) = 10$. This limit is called Amdahl's law.

Agents add a cost that this argument leaves out. Every subagent needs a brief, its results must be read and reconciled, and the orchestrator's own context grows with each one. We model that as a fixed overhead $c$ per agent, also in units of the single-agent time. In this section $c$ means only this overhead.

In plain words, the best number of subagents grows only like the square root of the parallel work per unit of overhead. At the optimum the two costs that depend on $n$ are equal. The coordination time $cn^\star$ and the leftover parallel time $f/n^\star$ are both $\sqrt{fc}$. Note also that one subagent is slower than none, since $t(1) = 1 + c$. That is why a system that spawns subagents for simple queries loses.

The dashed line is the overhead-free ceiling of 10. Even a small overhead bends each curve over, and the optimum moves left quickly as $c$ grows, from 17 agents at $c = 0.003$ to 5 at $c = 0.03$.

This is why Anthropic's research system writes scaling rules into its prompts. A simple fact gets one agent with 3 to 10 tool calls. A direct comparison gets 2 to 4 subagents. Only complex research with clearly divided responsibilities gets more than 10. Their early versions spawned 50 subagents for simple queries, which is the far right of these curves.

The widget adds the token bill. Slide the overhead $c$ and watch the best $n$ move, while the token count keeps rising in a straight line.

In practice, size the pool from a measured $f$ and $c$, not from how many workers the framework allows. In an interview, "how many workers?" is a derivation question. Write $t(n)$, set its derivative to zero, check the two whole numbers around $\sqrt{f/c}$, and say that tokens grow linearly while time falls.

from math import sqrt

def speedup(n, f, c):
    return 1.0 / ((1 - f) + f / n + c * n)

def best_n(f, c):
    n_star = sqrt(f / c)
    return n_star, round(n_star)              # placeholder: rounding is not always right

Scheduling subtasks that depend on each other

Real subtasks are rarely an even split. Some depend on others, as in "find the companies, then research each one", and they take different times. §24.4 treated this as scheduling a directed graph of tasks with no cycles, in which an arrow from $u$ to $v$ means $v$ needs $u$'s result and task $v$ takes time $w_v$. Two numbers bound any schedule. The total work is $W = \sum_v w_v$. The critical path length $L_{\text{cp}}$ is the longest total duration along any chain of dependencies, and even unlimited workers cannot finish sooner.

Now allow at most $C$ workers at a time. A greedy list schedule never leaves a worker idle while some task is ready to run. Its finishing time, the makespan, lies between $\max(W/C, L_{\text{cp}})$ and $W/C + (1 - 1/C)L_{\text{cp}}$ (Graham 1966). So greedy is always within a factor $2 - 1/C$ of the best possible schedule. For example, take ten independent one-minute tasks plus a chain of three one-minute tasks that must run in order. Then $W = 13$ and $L_{\text{cp}} = 3$, so on $C = 4$ workers any greedy schedule finishes between 3.25 and 5.5 minutes.

An orchestrator with a concurrency limit is a list scheduler, and the next exercise builds one. Its workers are simulated durations, not models. The scheduling is real and only the work is pretend.

def run_orchestrator(tasks, k, overhead=0.0):
    # placeholder: runs everything at once, ignoring k, dependencies, and failures
    end = max((t["duration"] + overhead for t in tasks), default=0.0)
    return {"makespan": end, "results": [t["result"] for t in tasks],
            "status": ["done"] * len(tasks), "log": [(t["id"], 0.0, t["duration"] + overhead, True) for t in tasks]}

Communication cost depends on the wiring

Agents talk by putting text into each other's contexts. Every token of that text is read, and paid for, by the receiver. So the communication graph sets the cost. Here is the smallest case. Three agents debate for one round with 100-token answers. Each agent reads the other two answers, 200 tokens, so the three read 600 tokens in all.

In general, write $\ell_{\text{msg}}$ for the length in tokens of one peer message and $\ell_{\text{sum}}$ for the length of one worker's summary.

Proof. In each round of debate, each of the $n$ agents reads $n - 1$ messages of $\ell_{\text{msg}}$ tokens. That is $n(n-1)\,\ell_{\text{msg}}$ per round and $n(n-1)\,\ell_{\text{msg}}R$ over $R$ rounds. Suppose instead that revision round $r$ shows the others' answers from every earlier stage, the first answers and revision rounds 1 to $r-1$. Then it shows each agent $r$ sets of $n-1$ answers, round $r$ costs $r\,n(n-1)\,\ell_{\text{msg}}$, and $\sum_{r=1}^R r = R(R+1)/2$. The star and the chain cost one read per edge. ∎

In plain words, when everyone talks to everyone, the reading grows with the square of the number of agents. When a hub reads summaries, it grows linearly.

These counts are per question. Like everything in an agent loop, they are read again on every later turn unless the context is managed. That is the quadratic growth of context from §24.1, where a loop of $T$ turns reads a number of tokens that grows like $T^2$. §24.6 prices tokens like these with and without caching.

In practice, a design whose agents all read each other needs a reason to pay quadratic reading, and a hub is the default. In an interview, expect to estimate the peer tokens of a proposed design on the whiteboard, as in the example above.

Context isolation is the benefit, lost decisions are the cost

The token counts say a star is cheap. The deeper question is what each agent knows. Here the two June 2025 posts disagree in a useful way.

Anthropic's research system works because subagents start with empty contexts. Each gets a brief, searches and reads on its own, possibly tens of thousands of tokens, and returns a condensed summary. The lead's context grows by the summary, not by the exploration. §25.2 works out the token savings of exactly this isolation.

Anthropic analyzed its system on BrowseComp, a benchmark of hard-to-find facts on the web. Three factors explained 95% of the variance in performance. Token usage alone explained 80%, and the number of tool calls and the choice of model explained the rest. In their reading, multi-agent systems work mainly because they let the system spend enough tokens on the problem, and separate context windows add room for parallel reasoning.

Cognition's point is the other side of the same coin. A subagent that sees only its brief does not see the decisions the lead made while writing it. Nor does it see the decisions other subagents are making at the same time. Their example asks for a clone of the game Flappy Bird, split into two subtasks, a moving background with pipes and a bird that moves up and down. One subagent builds a background in the style of Super Mario Bros. The other builds a bird that looks and moves nothing like Flappy Bird's. The final agent is left to combine two misunderstandings.

The obvious fixes fall short. Copying the original task into each brief does not help much. A real conversation runs many turns, and the lead's earlier tool calls and choices never make it into the brief. Even when each subagent sees the full context of the agents before it, two subagents working in parallel still cannot see each other's choices. The bird and the background come back in different visual styles. Cognition also noted that Claude Code, as of June 2025, never ran a subagent in parallel with the main agent. It usually asked its subagents only to answer a well-defined question, not to write code.

The two positions fit together once you look at what each task writes. Research reads a lot, and its pieces are nearly independent. A subagent's job is to find facts and report them. Its output is easy to check against sources, and two subagents researching different companies cannot make conflicting decisions. Coding writes a lot, and its pieces are coupled. Every edit carries choices, such as names, interfaces, and style, that other edits must match.

So isolation wins when subtasks are read-only, separable, and checkable. A single agent with one continuous context wins when subtasks share state. Anthropic's post says the same from its side. Domains in which all agents must share context, or which have many dependencies between agents, are not a good fit today, and most coding tasks have fewer truly parallel pieces than research.

Anthropic's engineering lessons read as countermeasures to the failures in this section.

• Each brief carries an objective, an output format, guidance on tools and sources, and clear task boundaries. Vague briefs such as "research the semiconductor shortage" led subagents to repeat each other's searches and to leave gaps.
• Effort-scaling rules in the prompts keep simple queries from getting many subagents.
• A separate citation agent ties every claim in the final report to a specific source, which is a verification step.
• The lead ran its subagents synchronously, waiting for each batch to finish. Anthropic named this as a bottleneck and an open problem, because agents that run asynchronously bring their own problems of coordination and consistency.

A different reason to split agents is security. The dual LLM pattern of §19.3 is a two-agent system whose wiring is the point (Willison 2023). A quarantined model reads untrusted text, such as web pages and emails, and returns only opaque values, such as a variable name standing for an extracted address. A privileged model plans and calls tools but never sees that text. The communication graph is chosen to cut the path an injected instruction would need, and the protocol (values, never instructions) is the defense.

In an interview, "Anthropic says yes and Cognition says no, who is right?" tests this reconciliation. Name the task property, read-heavy and separable against write-heavy and coupled, and say which design each one gets.

Roles and specialization

The early multi-agent frameworks gave agents human job titles and wired them into fixed procedures. Each contributed one idea that outlasted the framework.

Two cautions apply to roles. A role prompt changes what an agent pays attention to and how it formats its output, which helps separate concerns and add checks between stages. It does not give a model knowledge it lacks. Copies of one model in different roles still share the correlation $\rho$ of the voting part. Roles also add coordination, which is the cost the next part measures.

Why multi-agent systems fail, measured

Two studies turned the anecdotes into data.

Cemri et al. (2025) built MAST, a taxonomy of how multi-agent systems fail. A trace is the full log of one run, every message and tool call. They derived the taxonomy from 150 traces annotated by experts, whose agreement was high (Cohen's kappa of 0.88, where 1 is perfect agreement after correcting for chance). Then a model prompted with the taxonomy (OpenAI's o1, checked against the experts' labels at a kappa of 0.77) labeled 1,642 traces from seven frameworks, including MetaGPT, ChatDev, and AG2, the successor to AutoGen. The systems failed on 41% to 86.7% of their tasks. The failures fell into 14 modes in three categories.

• System design issues, 44.2% of failures. Repeating steps (15.7%), not recognizing that the task is complete (12.4%), disobeying the task specification (11.8%), losing conversation history, and disobeying a role.
• Misalignment between agents, 32.3%. A mismatch between an agent's reasoning and its action (13.2%), drifting off the task (7.4%), proceeding on wrong assumptions instead of asking for clarification (6.8%), conversation resets, ignoring another agent's input, and withholding information.
• Task verification, 23.5%. Incorrect verification (9.1%), missing or incomplete verification (8.2%), and stopping too early (6.2%).

Simple fixes guided by the taxonomy helped but did not solve the problem. On 32 small programming tasks with the same model, ChatDev succeeded 25.0% of the time. Clearer role specifications raised that to 34.4%. Making the work loop back until the technical lead confirms every review is satisfied, a check of the task's high-level objective, raised it to 40.6%.

Kim et al. (2025) ran a controlled comparison. They tested 260 configurations across six agent benchmarks, five architectures, and three model families, with compute matched. The architectures were a single agent and four kinds of multi-agent system. These were independent agents whose outputs are merged, a central coordinator, peers that talk without a coordinator, and a hybrid of the last two. Performance relative to a single agent ranged from +80.8% on financial analysis that splits into pieces to −70.0% on sequential planning.

The spread by task is the lesson. On the financial benchmark (Finance-Agent) every multi-agent architecture gained 57% to 81%. On planning in a Minecraft environment (PlanCraft) every one lost 39% to 70%. On SWE-bench Verified, a set of real GitHub issues checked by humans (§24.5), every multi-agent variant was slightly worse than one agent, by 2% to 15%.

The error amplification is the compounding of errors from §24.1 in a new place. There, a task of many steps succeeds only if every step does, so a per-step success rate $p$ gives $p^T$ over $T$ steps. Here, an answer built from many subresults is right only if every subresult is.

In practice, every design that merges the work of several agents needs a check before the merge, by code where possible and by a model where not. An interviewer will ask where your design verifies, and what fraction of bad subresults that check catches.

When several agents beat one

Putting the pieces together gives a decision procedure rather than a slogan. Read each row as a question to ask about the task in front of you.

The arithmetic of the section says the same things in numbers.

A multi-agent design earns its place when the task sits on the right side of each of these.

An orchestrator with the real SDK

In a real system, the orchestrator and the workers are model calls. Here is the skeleton of an orchestrator, written with the asynchronous client of the Anthropic Python SDK. It plans subtasks, runs one worker per subtask in parallel with a concurrency limit, and combines the results. A production system would also give the workers tools such as search and fetch, run each as a full agent loop as in §24.1, verify their summaries, and enforce budgets on tokens and time.

Each worker starts with only its brief, which is the context isolation that keeps the star cheap, and the lead sees only the summaries. The brief's three fields (objective, output format, boundaries) put Anthropic's lessons on briefs into code. The semaphore, a counter that lets at most max_workers calls run at once, is the concurrency limit $C$ of the orchestrator in Exercise 24.7c. The synthesis prompt asks the lead to flag conflicts between workers, a small defense against decisions that one worker made and another never saw.

In PyTorch

Mapping where debate beats an equal-compute vote is a natural batched computation. Each point $(\alpha, \beta)$ of a grid gets its own transition matrix, built from binomial probabilities, and all of them are run forward with batched matrix products.

The grid confirms the picture from the cell. For three agents at $p = 0.6$ and two rounds, debate beats the 9-answer vote on about 27% of the $(\alpha, \beta)$ square. It never does when $\alpha \le \beta$, and it does only where $\alpha$ exceeds $\beta$ by at least about 0.18. Right answers must be markedly more persuasive than wrong ones.

Read more

• Anthropic (2025). How we built our multi-agent research system. The orchestrator-worker design behind Claude's Research feature, with the 90.2% result, the token analysis, and concrete lessons on briefs, effort scaling, and evaluation.
• Yan (2025). Don't Build Multi-Agents. Cognition's case for sharing full context, with the Flappy Bird example and the two principles quoted above.
• Du et al. (2023). Improving Factuality and Reasoning in Language Models through Multiagent Debate. The debate protocol and its headline table. Read it alongside the two papers that control for compute.
• Smit et al. (2024). Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs, and Choi et al. (2025). Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models?. Debate against self-consistency and other ensembles with their costs, and the martingale argument that the vote does most of the work.
• Li et al. (2024). More Agents Is All You Need, and Wang et al. (2024). Mixture-of-Agents Enhances Large Language Model Capabilities. Sampling and voting as a scaling axis, and the evidence that diversity across models beats repeated samples of one.
• Liang et al. (2023). Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. Degeneration of thought in self-reflection, and debate with a judge as a way to keep disagreement alive.
• Cemri et al. (2025). Why Do Multi-Agent LLM Systems Fail?. The MAST taxonomy of 14 failure modes and the annotated traces behind it.
• Kim et al. (2025). Towards a Science of Scaling Agent Systems. The controlled comparison of architectures across benchmarks, with capability saturation and error amplification.
• Wu et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, and Hong et al. (2023). MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework. Two of the frameworks that defined conversation patterns and role pipelines.

Watch next

▶

▶