In this video I gave the exact same coding job to one Claude agent and to a team of five (a lead plus four sub-agents), then ran hidden tests to see who actually did better. I explain what a sub-agent is under the hood, what Anthropic, Cognition and Google researchers found, our full scoreboard, and some simple rules for when to split a job. It's for developers who use AI coding agents and want to know if sub-agents are worth the tokens.
What I cover in this video
- 0:00 One agent vs a team of five, and the headline result
- 0:46 What a sub-agent actually is: brief, start cold, work, report back
- 2:17 What Anthropic, Cognition and Google researchers found
- 4:36 How we set up the test: same model, same prompt, hidden tests
- 5:22 Play by play of the team on both jobs
- 6:54 The scoreboard: time, bugs fixed and tokens
- 8:24 Why the team costs so much more
- 9:10 When to split a job, and when not to
What a sub-agent actually is
A sub-agent is a second agent that your main agent starts with a tool call. In Claude Code, the main agent calls a tool named agent and passes in a written brief. If you want the bigger picture of how those tool calls fit together, I break it down in How the Claude Code Agent Loop Works.
The sub-agent starts with a brand new, empty context window. It doesn't see your chat or anything the main agent already read. All it gets is the brief, its own instructions, its own tools, and your project's CLAUDE.md file if you have one. It reads files and runs commands, and all of those tokens land in its own window. When it's done, it hands back one summary.
So the process has four steps: brief, start cold, work, report back. The part most people miss is that the summary is the only thing that makes it back to the main agent. Anything the sub-agent learned that didn't make it into the summary is gone.
Claude Code's docs say to use one when a side task would flood your main conversation with search results, logs or file contents you won't look at again. Explore and Plan are built in and read only. By default a sub-agent can start sub-agents of its own, up to three layers deep, with up to 20 running at once. The promise is clean context and work running in parallel. The question is what it costs.
What the labs found
Anthropic built its research feature this way: a lead agent plans, then spins up three to five sub-agents that search in parallel. They report that this setup beat a single agent by 90.2% on their internal research eval, and that parallel work cut research time by up to 90% on complex questions. That's their own internal test, though. They also report that multi-agent systems use about 15x the tokens of a normal chat, and that on BrowseComp, token usage alone explained 80% of the variance in scores. So a big part of why the team wins is that it spends more. In the same post they say most coding tasks have fewer truly parallel parts than research.
Cognition went further. In Don't Build Multi-Agents, Walden Yan uses a Flappy Bird clone as the example: one sub-agent builds a background that looks like Super Mario Bros., another builds a bird that doesn't look like a game asset, and the lead is left gluing two misunderstandings together. His rule is that actions carry decisions. In their later update, the new rule is that extra agents should add thinking, not actions, and writes stay with one agent.
Google, DeepMind and MIT researchers ran the biggest test I could find, 260 setups. On a finance task that splits cleanly, a coordinated team scored up to about 81% better. On step-by-step planning, every team setup did worse, by 39 to 70%. On SWE-bench Verified, every type of team came in a little under a single agent on average (2 to 15% worse), but that's only 20 tasks. For more on what SWE-bench measures, see Coding Agent Benchmarks Explained.
Our test and the scoreboard
Both sides ran Claude Sonnet 5.5 at high effort in Claude Code, with the same prompt word for word, a fresh sandboxed folder, web search off, and hidden tests the agents couldn't see. We ran two rounds of each job. The solo agent had the agent tool turned off. On the team side, the lead had to split the job across exactly four parallel sub-agents, then review and merge their work.
- Job one, bug hunt: a working invoicing backend, 42 files, 16 planted bugs, plus some code that looks wrong but is actually fine. It splits nicely by file.
- Job two, client job: a landlord's email thread, and the agent builds a maintenance request portal as one HTML file, with 12 hidden checks. It doesn't split nicely because everything lives in one file.
On the bug hunt, the lead split the work into four file sets that didn't overlap. Every sub-agent started by reading the README and listing files, because they start cold, so the same project got read from scratch four times. On the client job, the lead spent almost a full minute planning how three chunks of code would fit into one file before starting a single sub-agent, then glued everything together and tested it in a browser.
Bug hunt results
- Time: solo 4:07 vs team 3:35 in round one, solo 3:56 vs team 3:29 in round two. Team about 12% faster.
- Planted bugs fixed: solo 13 and 15, team 14 and 14. That's 28 each. Neither side broke any of the 14 tests on code that already worked.
- Tokens: solo about 1.6 million, team about 3.7 million, so about 2.3x.
- Extra real problems the team reported that weren't on our list: five in round one, seven in round two. One example: on a real server, requests with no body came back as errors, so checking who you're logged in as just failed. The solo agent didn't report any of these in either round.
Client job results
- Every run passed all 12 checks, solo and team, both rounds.
- Time was a coin flip: the team was half a minute slower in round one and about a minute and a half faster in round two.
- Tokens: solo about 400,000, team about 2.2 million. Over 5x for the same result.
There are two reasons for the cost. Every sub-agent starts cold and has to read the project again, and the lead has to plan the split and then read and stitch together everything that comes back. You pay for that coordination on every job.
When to split a job, and when not to
Split when the parts don't depend on each other and the work is mostly reading: auditing a codebase by area, finding where something lives, researching a few options at once, or reviewing a pull request with fresh eyes. That's the kind of work where our team caught extra problems.
Don't split when it's one tightly connected thing: one file, one feature, or steps that depend on each other. That was our client job, with the same score for 5x the tokens. Claude Code's agent teams docs say something close: for step-by-step work, edits to the same file or lots of dependencies, one session or a couple of sub-agents works better than a whole team.
When you do split:
- Keep it to three or four agents.
- Write each brief for someone who has never seen the project, because that's exactly the situation.
- Ask for short reports, not essays.
- If more than one agent writes code, give each one its own working copy of the repo so they don't overwrite each other.
That last rule is how Swarm works in Beckon: every agent gets its own working copy, and you can watch each one running in its own pane.
Key takeaways
- A sub-agent gets a written brief and an empty context window, and only its summary comes back to the main agent.
- On the bug hunt, the team was about 12% faster and fixed the same 28 planted bugs as the solo agent, for about 2.3x the tokens.
- The team did find 5 and 7 extra real problems that the solo agent never reported.
- On a one-file job, everyone scored 12/12 and the team used over 5x the tokens.
- Split independent work that's mostly reading. Keep one tightly connected job with one agent.
Full video transcript
0:00We gave the exact same coding job to one claude agent and to a team of five, a lead and four sub agents. The team finished first both rounds. It also burned more than twice the tokens. And when we ran the hidden tests, the team and the solo agent fixed the exact same number of bugs, 28 each. So, were the sub agents worth it? Kind of. They found something the solo agent never did. And I'll show you exactly what. And on our second job, the team spent about five times the tokens for the same score. So, in this video, we're going to be breaking down what a sub aent actually is under the hood, what Anthropic, Cognition, and Google found when they tested this, our own head-to-head, the
0:46scoreboard, and some simple rules for when you should split a job and when you shouldn't. Let's jump right into it. So, what is a sub agent exactly? A sub agent is a second agent that your main agent starts with a tool call. That's it. In cloud code, the main agent calls a tool named agent and it passes in a written brief. Here's how it actually works. The sub agent starts with a brand new empty context window. It does not see your chat. It does not see what the main agent already read. All it gets is that brief, its own instructions, its own tools, and your project's claw.md file if you have one. Then it goes and does the work. It reads files. It runs commands. And every single one of those tokens lands in its own window, not
1:31yours. And when it's done, it hands back one summary. The main agent only ever sees that summary. So the four-step process is brief, start cold, work, and report back. And here's something most people don't think about. That summary is the only thing that crosses back into the main agent's head. Everything the sub agent learned that didn't make it into the summary is gone. Cloud Code's own docs say to use one when a side task would flood your main conversation with search results, logs, or file contents you won't look at again. Explore and plan are builtin and read only. And by default, a sub aent can even start sub aents of its own up to three layers deep and up to 20 can run at the same time. So that's the big promise. Clean context
2:17and work running in parallel. Now the question is what it costs you. What if a new video for your business showed up every morning? Nightcrew learns your business once, then makes a short every night in your own voice. It runs five brands right now, over half a million views a week. Nightcrew nightcrew.studio. So what did the labs find when they actually tested this? Anthropic built their research feature this way. A lead agent plans, then spins up three to five sub aents that search in parallel. Anthropic reports that a lead with sub agents beat a single agent by 90.2% on their internal research eval and parallel work cut research time by up to 90% on complex questions which is insane. Now that's their own internal test. So take that for what you will but
3:04there's a catch. Anthropic also reports that multi- aent systems use about 15 times more tokens than a normal chat. And on browse comp, token usage by itself explained 80% of the variance in scores. So a big part of why the team wins is that it simply spends more. And in that same post, Anthropic says most coding tasks have fewer truly parallel parts than research and that agents aren't great yet at coordinating with each other in real time. Cognition went even further. In June of last year, Walden Yan wrote a post called Don't Build Multi- Aents. His example is a Flappy Bird clone. One sub agent builds the background. Another builds the bird. The first one builds a background that looks like Super Mario Brothers. The second builds a bird that doesn't look like a game asset at all. And now the
3:50lead has to glue two misunderstandings together. His rule is that actions carry decisions. And when two agents make different decisions, you get bad results. Then in April, Cognition posted an update. Their new rule, extra agents should add thinking, not actions. Rights stay with one agent. and Google and MIT researchers ran the biggest test I could find, 260 setups. On a finance task that splits up cleanly, a coordinated team scored up to about 81% better. On step-by-step planning, every team setup did worse by 39 to 70%. And on S.Bench verified, every type of team setup came in a little under a single agent on average, somewhere between 2 and 15% worse. That one's only 20 tasks, so take
4:36it with a grain of salt. So, I wanted to see it for myself. Here's how we ran it. Same model on both sides. Claude Sonnet 5.5 at high effort running in Claude code. Same prompt, word for word. Fresh folder, sandboxed, web search off, and hidden tests. The agents can't see two rounds of each job. On the solo side, we turn the agent tool off. It does the whole thing itself. On the team side, the lead has to split the job across exactly four sub aents running in parallel, then review and merge their work. Job one is a bug hunt, a working invoicing backend we set up for the test. 42 files with 16 bugs planted in it and some code that looks wrong but is actually fine. This job splits up nicely
5:22because each agent can take its own set of files. Job two is a client job. A landlord's email thread and the agent has to build a maintenance request portal as one single HTML file. 12 hidden checks. Now, this one does not split nicely because everything lives in the same file. So, let's look at the team on the bug hunt. You can see the lead reads the readme, runs the tests, and about 6 seconds in, it says it's splitting by area into four file sets that don't overlap. Invoices and dates, payments and reports, off and the HTTP layer, storage, web hooks and reminders. Then all four briefs go out in about 8 seconds. And look at what each sub aent does first. It reads the readme and lists the files, all four of them,
6:08because remember they start cold. That's the same project getting read from zero four times over. About 2 minutes in, sub agent 3 reports back and it says it also found bugs in files that belong to the other agents. So now the lead is waiting on everyone, then checking the combined changes and writing it all up. Done in 3 minutes 35. Now the client job. This is a lot more interesting. The lead spends almost a full minute just planning before it starts a single sub agent. It's designing how three chunks of code are going to fit into one file. Then four jobs go out. the core logic, the page and the tenant tab, the manager tab, and the email back to the client. Then the lead glues the code together into one page, tests it in a browser,
6:54and you can see Chrome's headless mode crashes once on its own, so it just retries with different settings. Okay, so here's the scoreboard. Bug hunt. Round one, solo 4 minutes 7, team 335. Round two, solo 356, team 329. So, the team was faster both rounds, about 12% on average. Now, the hidden tests. Round one, solo fixed 13 of the 16 planted bugs. Team fixed 14. Round two, solo fixed 15. Team fixed 14. So, that's 28 each, dead even. And neither side broke any of the 14 tests on stuff that already worked. The tokens though, on average, the solo agent used about 1.6
7:39million tokens. The team used about 3.7 million. That's about 2.3 times the tokens for the same 28 bugs. But here's what the team did that the solo agent didn't. Probably because four agents each read their own area really closely. The team reported real problems that weren't even on our list. We checked everyone against the spec. Five in round one, seven in round two. Like this one. On a real server, requests with no body came back as errors. So checking who you're logged in as just failed. The solo agent never reported one of those either round. Now the client job. Every run passed all 12 checks. Round one and round two. Solo and team. Same score. Time was a coin flip. The team was half a minute slower in round one and about a
8:24minute and a half faster in round two. But tokens. The solo agent used about 400,000. The team used about 2.2 million on average. That's over five times the tokens to build the exact same thing. So why? Two reasons. One, every sub agent starts cold, so it pays to read the project again from scratch. Two, the lead has to plan the split, then read everything that comes back and stitch it together. You pay for that coordination on every single job. So here's how I would use this. Split when the parts don't depend on each other and the work is mostly reading. Auditing a codebase by area. Searching for where something lives. Researching a few options at once. Reviewing a pull request with fresh eyes. That's where we saw the team
9:10actually catch extra problems. Don't split when it's one tightly connected thing. One file, one feature, steps that depend on each other. That's our client job. Same score, five times the tokens. And Claude Codes's docs on agent teams say something really close. For step-by-step work, edits to the same file or lots of dependencies don't spin up a whole team. One session or a couple of sub aents does the job better. And when you do split, a few rules. Keep it to three or four agents. Write each brief like you're handing it to someone who has never seen the project, because that's literally what's happening. Ask for short reports, not essays. And if more than one agent is going to write code, give each one its own working copy of the repo so they're not overwriting each other. That's actually how Swarm
9:56works in Becken. Every agent gets its own working copy and you can see each one running in its own pain. So like I said at the start, the team was faster on the bug hunt. It caught extra problems the solo agent missed and it cost a lot more tokens every single time. Whether that's worth it really depends on the job. So how are you using sub agents right now? Do you split every job or do you keep one agent and let it cook? Drop your answer down below. And if this one saved you some tokens, leave a like so more builders end up seeing it. Have a blessed day.