Benchmark: Claude Code solo vs a 3-agent team — nearly 2x slower, slightly cheaper, but the winner on the big security task

A developer has published a rather rare kind of measurement: Claude Code working alone, head-to-head against a three-agent "team" — Claude as coordinating tech lead, the cheap pi agent writing code, Codex as independent reviewer — on the same Rust project, from the same base commit, graded by blind review. The results defy the usual expectations: the multi-agent workflow is not faster, in fact almost twice as slow; the money saved is thinner than imagined; but on the largest, security-sensitive task, the team's version was the only one that passed.

Quick answer

• On speed: the multi-agent team was slower on all three tests — the small task took ~28 minutes vs 19, the medium one 16.1 vs 8.7 minutes, the large one 66.2 vs 30.5 minutes (nearly twice as slow on the latter two).
• On cost: delegating the typing to a cheap model cut the Claude bill by about 25% (medium task: 1.54 USD vs 2.05 USD) and 15% (large task: 3.30 USD vs 3.90 USD) — but those figures exclude the Codex review calls, which the multi-agent branch consumed 2–3 times more of.
• On quality: on the small and medium tasks, both sides passed blind review (APPROVE). On the large MFA task, the team's version was APPROVED while Claude's solo version was marked CHANGES_REQUIRED over a leaked OpenAPI contract mismatch.
• Tiny sample: every conclusion comes from 3 test pairs on the author Brian Ng's own project — the author himself stresses these are "early signals", not firm conclusions.
• The author's current strategy: tier by risk — small tasks go to Claude solo, ordinary features get the cheap agent plus one review, and anything touching auth, databases or public APIs requires two independent reviews.

What does this benchmark measure, and how?

It compares two ways of organizing AI coding agents on the same existing Rust project:

• The solo branch: Claude Code alone (in the author's configuration: Opus 5.5, Medium effort) reads the requirements, writes the code and checks it itself.
• The team branch: Claude Code acts as Tech Lead, the pi agent running a cheap model (DeepSeek V4.1 Flash, High) is the main developer writing code, and Codex (GPT-6.1 Sol, High) is an independent reviewer cross-checking for errors. The author watched the whole process live through the Herdr TUI.

The measurement method was designed to limit bias — which is what makes this benchmark more readable than most:

• Same starting point: each task was split from the same base commit into two independent worktrees, with completely fresh sessions for each branch.
• Locked by a task contract: to prevent agents from "grading their own homework", every task carried a contract stating the goal, the list of files allowed to change, and the test command; a worker reporting "done" was not enough — the code had to pass an automated check script (a gate) to count as finished.
• Blind review: the two patches were anonymized as "Version X" and "Version Y" and graded by a reviewer who did not know which side wrote which.
• A fairness caveat: only Pair 1 is 100% Claude solo. In Pairs 2 and 3, the solo Claude branch still received a final Codex review pass, so those two pairs really measure a narrower question: how much money and time does pushing the typing to a cheap model save or cost? Costs were converted from token logs at list prices for relative comparison.

Who is on the 3-agent team?

Role	Agent / model (per author)	Job
Tech Lead	Claude Code (Opus 5.5 – Medium)	Understands the request, writes the task contract, coordinates and accepts the result
Main developer	pi (DeepSeek V4.1 Flash – High)	Writes the code, reads the docs, runs the tests — chosen for its low cost
Independent reviewer	Codex (GPT-6.1 Sol – High)	Advises and cross-checks for errors, writes no code

Results across three difficulty levels

Pair	Task	Time: team vs solo	Cost: team vs solo	Blind review
1	Add a version field to the health endpoint (small)	~28 min vs 19 min	Claude cost more than solo (24,737 vs 22,232 output tokens)	Both APPROVE
2	Add an admin CLI command, 6 files, ~400–500 lines (medium)	16.1 vs 8.7 min	1.54 USD vs 2.05 USD (−25%)	Both APPROVE
3	View and unlock MFA, ~25 files, security-sensitive (large)	66.2 vs 30.5 min	3.30 USD vs 3.90 USD (−15%)	Team: APPROVE — Solo: CHANGES_REQUIRED

Reading each pair closely shows the picture is not simply "the team lost":

• Pair 1 — splitting a tiny task is a pure loss. Coordination overhead dwarfs the task's value; the team run also hit infrastructure hiccups that forced the author to nudge the agents manually, inflating the numbers. The one plus: the multi-step process caught a missing unit test that the solo side had skipped.
• Pair 2 — money starts dropping, time still loses. With Claude only coordinating, its output tokens fell sharply from 33,009 to 18,730. Both versions worked, but pi's weaker model needed two review rounds to finish — in round one, Codex caught a real logic bug pi had missed.
• Pair 3 — the deciding task. On core security logic the two sides were equal. The difference was review depth: the solo version let an OpenAPI contract mismatch slip through its internal review, caught only at the final blind review; the pi version had two serious bugs blocked in the very first review round, and being forced to fix thoroughly across rounds left the final code cleaner. The author deployed the team's version to a real server: every scenario ran smoothly, except the OIDC flow, which was not yet tested. Claude's output tokens in this pair dropped from 63,439 to 37,595, and peak context fell from 240,402 to 183,060 tokens.

The three most valuable conclusions

1. Delegating to a cheap model does not automatically cut cost

Even after handing the typing to pi, Claude still consumed most of the bill. In Pair 3, a full 77% of the data Claude read was consumed before delegation even began — it had to read the entire codebase to have enough information to write the task contract. That data then sat in the context window, so every later conversation turn paid to re-read it. Add that the pi branch consumed 2–3 times more Codex reviews than the solo branch (not counted in the 15–25% savings), and the real saving is far thinner than first expected.

2. One AI review pass is not enough for sensitive features

Same model, same prompt template — yet one reviewer waved through bugs that another caught, a phenomenon visible within just these three pairs. For anything touching authentication, data structures, migrations or public APIs, the author now mandates two independent review passes on the exact final code before merging.

3. More agents mean more operational burden

The more tools you wire together, the more failure points: from hook scripts to an agent reporting "done" while actually hanging on a hidden prompt — each small glitch broke one of the author's runs at some point. Even the automated gate script the author wrote himself had bugs, and needed seven Codex review rounds to fully fix.

The author's strategy: tier by risk level

Because he values code safety and cost over waiting time, the author keeps the multi-agent workflow as his default — but tiered rather than applied mechanically:

• Small task, fix location already clear: Claude works alone, at most one independent review, skip the heavy codebase reconnaissance.
• Ordinary feature task: hand all codebase scouting and coding to pi, plus one review pass.
• Anything touching auth, database or public APIs: Codex cross-checks from the contract design stage, then two independent reviews at the output.

His next tuning direction: forbid Claude from reading code directly too much, pushing that reconnaissance to the cheaper agent with a target of cutting at least another 30% of cost; and consider downgrading the coordinating model from Opus to Sonnet to halve its price — remeasured results to follow.

Limits of this benchmark

To avoid over-reading the results, keep the limits the author acknowledges or that the method itself implies:

• Sample size n=3 on a single Rust project with a single operator — enough for signals, not enough to generalize to every team and language.
• The solo branch is not fully "solo" in Pairs 2 and 3 because of the final Codex review, so those pairs compare "who types the code", not "review vs no review".
• Pair 1's numbers are noisy due to a mid-run infrastructure failure (agents had to be nudged by hand), inflating time and tokens.
• Costs are relative conversions from token logs at the most expensive model's list price, without fully counting Codex across every review round of the team branch.

Frequently asked questions

Are multiple AI agents faster than one agent?

In this measurement: no. The multi-agent team was slower on all three tasks — from 1.5x to more than 2x — because coordination, hand-offs, review loops and infrastructure incidents ate the parallelism advantage. Multi-agent trades speed for verified quality and part of the cost.

Why does delegating to a cheap model only save 15–25%?

Because the most expensive part is not typing code but reading and understanding: the coordinating model must load nearly the whole codebase to write a correct task contract (77% of reading in Pair 3 happened before delegation), and that data keeps being billed on every subsequent turn because it sits in the context window.

What is blind review in this benchmark?

An objectivity device: the two patches for the same task are anonymized ("Version X", "Version Y") and graded by a reviewer who does not know whether the solo branch or the team branch wrote each one — eliminating bias toward a model's "brand".

When should you use a multi-agent workflow?

When the task is large or sensitive enough that review cost is worth it: features touching auth, databases, migrations, public APIs. Conversely, for small tasks with a clear fix location, one strong agent working alone — with at most one independent review — is decisively more economical.

What do the task contract and the automated gate do?

The contract locks the work's scope (goal, files allowed to change, test command) so agents cannot wander or self-declare completion; the gate script is a mechanical checkpoint the code must pass before it counts as done — two mechanisms against AI agents' habit of grading their own homework.

Conclusion

The value of this benchmark is not the win-loss scoreline but how it exposes three rarely stated truths about multi-agent hype: coordination has a price, a cheap model does not magically cheapen the invoice, and real quality comes from layers of independent review rather than from the number of agents. For teams building AI coding workflows, the author's risk-tiered formula — solo for the small, many layers of verification for the large and sensitive — is a far more practical starting point than "multi-agent for everything".

Source: Brian Ng's benchmark post on Facebook (3 Nov 2026), edited and structured by Wotech Electronics. All figures are the author's measurements on a personal project, not official data from any model provider.