In this video I put three Anthropic models, Fable 5.1, Sonnet 5.5 and Haiku 5.5, through four real coding jobs inside Claude Code: a game build, a freelance client job, a redo of a viral demo and a hard bug hunt. It's for developers and builders who run AI coding agents every day and want to know which model to reach for, and why the speed on a leaderboard doesn't tell you how fast a model finishes your job.
What I cover in this video
- 0:00 Haiku finished the client job first, then the hidden tests confirmed it
- 0:45 The contestants and their outside numbers from Artificial Analysis
- 3:05 How we ran it: effort levels, the rules and "time to green"
- 3:50 Test 1: a Call of Duty style shooter in one HTML file
- 5:22 Test 2: the landlord client job and the clock
- 6:07 Test 3: redoing a viral code-made short
- 7:38 Test 4: the Tally House bug hunt
- 9:10 The scoreboard, the charts and my rule of thumb
The lineup and the outside numbers
Anthropic positions these three very differently. Fable 5.1 (out in September) is called its most advanced model for coding and knowledge work, and its own docs label its speed as slower. Sonnet 5.5 (September 28) is pitched as the best combination of speed and intelligence. Haiku 5.5 (October 7) is its fastest model to date at standard speed, built for high volume work and suited to running as a sub agent.
As of November 6, Artificial Analysis measured output speed at about 239 tokens per second for Haiku at max, about 151 for Sonnet at max and about 86 for Fable at xhigh. On paper Haiku is almost three times faster than Fable. Their Intelligence Index puts Sonnet at 55, Fable at 50 and Haiku at 36. The number most people skip is how many tokens each one writes to get through that index: Fable about 115M, Haiku about 298M, Sonnet about 335M. How fast a model types and how fast it finishes are two different things, and our bench showed the same pattern. If you want more background on how these leaderboards work, I wrote up coding agent benchmarks explained.
How I ran the tests
Every model ran as a coding agent in Claude Code with the same setup each time:
- The same prompt, word for word
- A fresh empty folder every run
- Web search off, no follow ups, no nudging
- Sandboxed so the agent couldn't see the hidden tests
Effort levels were Fable at xhigh, Sonnet at max and Haiku at max, which are the same settings Artificial Analysis measured. (For how effort levels change behavior, see Claude Code effort levels tested.) The client job and bug hunt each ran three rounds and I report the median.
The part I care about most: every 10 to 20 seconds we saved a copy of each folder, then ran the hidden tests on every copy afterward. That tells us the minute the work actually passed, not just when the model said it was done. Time to green, not time to done.
The four tests
Test 1: the game
The task was a Call of Duty style shooter in one HTML file: WASD to move, mouse look, shooting, reloading, and enemies that take cover and shoot back, with a 45 minute limit. Every game was played with the same inputs by our recorder. Haiku finished in 34:58 with a red mesa town at sunset and a full ammo counter. Fable finished in 35:56 with a desert market, crates, sandbags and a gun that kicks. Sonnet thought for about 11 minutes, then built the game as 14 separate files in a scratch folder outside its own folder with a build script, and was still testing the gun when time ran out. Its folder was empty. I show that scratch build in the video, labelled. Honestly it's the best looking of the three, but it never shipped, so it counts as out of time.
Test 2: the client job
A small landlord sends a messy email thread about a maintenance request portal. Requirements are spread across emails, one changes halfway through and one is unclear. The agent builds the app and writes the reply. Twelve hidden tests check every requirement, including the changed one. All three models passed 12 of 12 in every round, so quality was a tie. The clock was not:
- Median time to green: Haiku 7:19, Fable 9:41, Sonnet 15:23
- Median time to done: Haiku 13:41, Fable 14:18, Sonnet 37:42
Sonnet had a working app and then spent over 20 more minutes checking time zones and weekends and polishing. Two third party judges scored the reply emails blind against a rubric, and every one got 9 or 10 out of 10. All three told the client the same honest thing: the page can't send a text message on its own.
Test 3: the viral redo
We rebuilt a popular X post: a short made entirely in code with Sonnet 5.5. The creator never shared the prompt, so we wrote our own: a 20 second vertical short about octopuses, all in code, with pop-up captions and motion graphics. Fable finished in 18:37, Haiku in 24:29. Sonnet put a page in its folder at about 30 minutes and was still reworking it when the 45 minutes ran out. What it left is beautiful, but on both creative builds at max, Sonnet didn't finish.
Test 4: the bug hunt
Tally House is an invoicing and client portal backend we built: about 3,000 lines across 42 files, with 16 hidden bugs and 11 decoys (code that looks wrong but isn't). The prompt never says how many bugs there are. Hidden tests decide what's fixed, 14 more tests catch regressions, and judges dock a point for every made-up bug.
- Bugs fixed (median, of 16): Sonnet 14, Haiku 14, Fable 10
- Points (of 32): Sonnet 27.5, Haiku 24.5, Fable 20
- Regression tests broken: 0 in all 9 runs
- Median time to done: Fable 15:54, Haiku 23:05, Sonnet 35:54
Sonnet won on points because it explained its fixes and the judges found almost nothing made up. One judge said Haiku reported a few things that aren't actually bugs. All three spent the first 10 to 20 minutes mostly reading, then landed their fixes almost all at once.
Why the slowest typist finished first
The chart that matters is time to green versus time to done. The gap is the model still working after the tests already pass: about 6 minutes for Haiku, about 4 and a half for Fable, and most of the run for Sonnet at max. Output tokens explain the rest. On the client job, Fable wrote about 70K, Haiku about 169K and Sonnet about 255K. On the bug hunt, Fable wrote about 77K while Haiku wrote about 308K, four times as much. Haiku types almost three times faster than Fable, but it writes so much more that Fable can still finish first. Claude Code's own docs say max is prone to overthinking, and we watched it happen. Sonnet at max finished last in all 8 timed runs.
Key takeaways
- Measure time to green, not leaderboard speed. What decides the clock is how much a model writes and the effort level you pick.
- If the job has tests the agent can run (a client feature or a sub agent task), start with Haiku at max and let the tests tell you when it's done.
- If you want it fast and you'll review it yourself, use Fable at xhigh. It wrote the fewest tokens and finished the bug hunt first, but missed the most bugs.
- For the bug that's been fighting you all week, use Sonnet at max, then go do something else while it works.
- For business work, all three built a working client app and wrote an honest reply. Haiku just got there first.
Full video transcript
0:00Across three rounds, Haiku 5.5 finished the client job first. Then we ran the hidden tests and every single one passed in about 7 minutes. Sonnet 5.5 got the same app passing at about 15 minutes and then it kept going for over 20 more minutes before it said it was done. And on the game, Sonnet built the best looking shooter of the three. You can see it right here. It never put it in its folder. The clock ran out first, so it doesn't count. So, in this video, we're going to be breaking down Fable 5.1, Sonnet 5.5, and Haiku 5.5 on four real coding jobs. The outside numbers, the race, the scoreboard, and the charts. And at the end, I'll give you a simple rule for which one to run on
0:45which job. Let's jump right into it. So, here's the lineup. Three Anthropic models with very different speeds. Fable 5.1 came out in September. Anthropic calls it the world's most advanced model for coding and knowledge work and their own docs label its speed as slower. Sonnet 5.5 came out September 28th and Anthropic calls it the best combination of speed and intelligence. And Haiku 5.5 came out October 7th. Anthropic calls it their fastest model to date at standard speed built for high volume work. And they say it pairs well as a sub aent on coding work. Now here's what this is all about. As of November 6th, artificial analysis measures Haiku at max at about 239 tokens a second. Sonnet at max is
1:33about 151 and Fable at X high is about 86. So on paper, Haiku is almost three times faster than Fable. But look at their intelligence index. Sonnet at max scores 55. Fable at XHigh scores 50. Haiku at max scores 36. So on their index, the smartest of the three isn't the one Anthropic calls the most advanced, which is pretty interesting. And here's something most people don't think about. Artificial analysis also counts how many tokens each one writes to get through their whole index. Fable at XHigh writes about 115 million. Haiku at max writes about 298 million. Sonnet at max about 335 million. So, how fast a
2:20model types and how fast it finishes are two different things. Keep that in mind because our bench shows the exact same thing. Want to make videos like this one with Claude Code? That's what I teach. Step by step inside Shipping School Premium. New lessons every single day, live calls every week, a full course library, and over 500 hours of replays. 263 builders are already in. Join free link below. So, here's how we ran it. Every model runs as a coding agent in claude code, which is Anthropic's own tool. Same prompt, word for word. A fresh empty folder every time. Web search off. No follow-ups and no nudging. And each one is sandboxed so it can't see the hidden tests. We picked one effort level for each model. Fable
3:05at Xigh, Sonnet at max, and Haiku at max. Those are the same settings artificial analysis measured, so at least we're comparing the same settings. There are four tests. a game build because I want to know if it can make a game. A real client job, which is the business question, a redo of a viral demo from X, and a really hard bug hunt. The client job and the bug hunt ran three separate rounds each, and the scoreboard shows the median. And here's the part I care about the most. Every 10 to 20 seconds, we saved a copy of what was in each folder. Then afterwards, we ran the hidden tests on every copy. So, we don't just know when a model said it was done. We know the minute its work actually passed. Time to green, not time to done. So, test one. Build a Call of
3:50Duty style shooter in one HTML file. WD to move, mouse look, shooting, reloading, enemies that take cover, and shoot back. 45 minutes. And every game gets played with the exact same inputs by our recorder. So, nobody gets a better driver. Haiku finished in about 35 minutes. A red mea town at sunset, palm trees, a full ammo counter, and it plays. Fable finished in about 36. A desert market with crates and sandbags and a gun that kicks. Both of them work. And Sonnet Sonnet looked at the empty folder and then it thought for about 11 minutes straight. Then it built the game in pieces. 14 separate files in a scratch folder outside its own folder with a build script to glue them together. and it was still testing the
4:36gun when the 45 minutes ran out. So, its folder was empty. Zero files. Now, I did go grab what it had built in that scratch folder, and this is it. Look at that lighting. Honestly, it's the best looking game of the three, but it never shipped it. If that was a client, they got nothing. So, on the scoreboard, Sonnet ran out of time. Which one looks better to you? Comment down below. So test two is the one I care about for business, a real freelance job. A small landlord sends a messy email thread about a maintenance request portal for their building. The requirements are spread across a bunch of emails. One of them changes halfway through, and one of them is unclear. The agent has to build the app and write the reply email back to the client. 12 hidden tests check
5:22every requirement, including the one that changed. And all three models passed all 12 in every round. Every one of them caught the change. So on quality, it's a tie. Here is what I'm talking about, though. Look at the clock. Median time to green. Haiku 7 minutes 19 seconds. Fable 941. Sonnet 1523. And then median time to done. Haiku 1341, Fable 1418, and Sonnet 3742. So, the fast one really did finish first, and it was right. And Sonnet had a working app and then spent over 20 more minutes checking time zones and weekends and polishing. The tests were
6:07already passing, and the reply emails were really good across the board. Two third party judges, Grock 4.7 and Quen 3.8 8 Max Prime scored them against a rubric without knowing which model wrote which. Every email scored 9 or 10 out of 10. And all three told the client the same honest thing. The page can't send a text message on its own, which is exactly what I'd want my own developer to say. So for test three, we looked at what people are actually building with these models on X. The most viewed one was a crypto trading scanner built with Fable and GPT6 Astra, but that needs live market data, so there's no fair way to rerun it. The next one down came from a creator on X at YouTube Priest01, a
6:52short made entirely in code with Sonnet 5.5 on a phone with 4 gigs of RAM. As of November 6th, it had about 30,000 views. He never shared his prompt, so we wrote our own. A 20-se secondond vertical short about octopuses, all in code with pop-up captions and motion graphics. Fable was done in about 19 minutes. Haiku in about 24. Both look like something a real creator could post. Haiku drew a little cartoon octopus with iron and copper tiles for the blood. Fable did humans versus octopus, iron versus copper, which is pretty clever. And Sonnet, Sonnet ran out of time again. It did put a page in its folder at about 30 minutes and it was still reworking it when the clock ran out. So, here's what it left. And honestly, it's
7:38beautiful. But on both creative builds at max, sonnet didn't finish. So, test 4 is the hard one. A codebase we built called Tally House. It's an invoicing and client portal backend. About 3,000 lines across 42 files with 16 hidden bugs and 11 decoys. Decoys are code that looks wrong but isn't. And the prompt never says how many bugs there are. Hidden tests decide what's fixed and 14 more tests catch any fix that breaks something else. Then the same two judges check the write-ups and you lose a point for every bug you made up. And this is where Sonnet earns its keep. Sonnet and Haiku both fixed 14 of 16 in the median round. Fable fixed 10. And not one model
8:24broke a single thing in any round. But on points, Sonnet wins. 27 1/2 out of 32 because it explained what it fixed. And the judges found almost nothing made up. Haiku got 24 1/2. It explained a little less. And one of the judges, Grock, said it also reported a few things that aren't actually bugs. Fable got 20. Now look at how they did it. This chart is bugs fixed over time. All three spend the first 10 to 20 minutes mostly reading zero fixes and then the fixes land almost all at once inside about a minute. In round one, Haiku started fixing at about 11 minutes, Fable at about 13 and Sonnet at about 20. And the clock median time to done, Fable about
9:1016 minutes, Haiku about 23, Sonnet about 36. So Sonnet was the best bug hunter here. And it took more than twice as long as Fable to get there. So here's the scoreboard. All the medians in one place. On the client job, everybody passed 12 out of 12. Haiku was green first, then Fable, then Sonnet. On the bug hunt, Sonnet scored the most points. Haiku tied it on bugs fixed, and Fable was done first. On the game and the viral short, Haiku and Fable both delivered, and Sonnet ran out of time on both. And here's the API equivalent cost of a run, which is what Claude Code reports. We ran it on a subscription. On the client job, Haiku was about 60, Sonnet about $5, and Fable about 6. On
9:56the bug hunt, Haiku about $2, Sonnet about $650, and Fable about $750. And this is the chart that matters. Time to green versus time to done. The gap between those two bars is the model still working after the tests already pass. On Haiku, it's about 6 minutes. On Fable, about 4 and a half. On Sonnet, at max, it's most of the run. And tokens tell the same story artificial analysis told us at the start. On the client job, Fable wrote about 70,000 tokens. Haiku about 170,000 and Sonnet about 255,000. On the bug hunt, Haiku wrote about 300,000 and Fable about 77,000. That's four times as much. So, Haiku types
10:41almost three times faster than Fable, but on the bug hunt, it wrote four times more. That's why the slowest typist finished first. So, let me answer the four questions. Can it make a game? Haiku and Fable, yes, in about 35 minutes each. Sonnet at max built the nicest one, but it didn't deliver it in 45 minutes. Can it hunt bugs? Yes. All three found real bugs without breaking anything. Sonnet scored the highest. Haiku fixed just as many, but one judge said it made a few up. Fable was the fastest and it missed the most. What does it mean for AI coders? The speed on the leaderboard is how fast a model types. It's not how fast it finishes your job. What matters is how much it writes and the effort level you pick.
11:27And Max can overthink. Cloud Code's own docs say it's prone to overthinking, and we watched it happen twice. And how would you use it in a business? The client job is the proof. All three built the landlord a working app that passed every test and wrote an honest email back. Haiku just got there first. So, here's my rule of thumb. If the job has tests the agent can run, like a client feature or a sub agent task, start with Haiku at max and let the tests tell you when it's done. If you want it done fast and you'll review it yourself, Fable at X high. And for the bug that's been fighting you all week, sonnet it at max. but kick it off and go do something else because it is going to take a while. So, which one are you running as your daily driver right now? And how often do you
12:12catch your agent still working after the tests already pass? Tell me in the comments. And if this one helped you pick, a like goes a long way. Have a blessed day.