beckon
Beckon blog

Gemini 4 Neon vs Claude Sonnet 5.5: Coding Agent Test Results

ยท Beau Johnson

In shortIn my four coding tests inside Claude Code, Gemini 4 Neon and Claude Sonnet 5.5 came out almost tied on quality. Sonnet was faster on three of four (35.3 minutes total versus 50.5), and Neon's bill was more than twice Sonnet's API-equivalent cost.

In this video I put Google's new Gemini 4 Neon up against Claude Sonnet 5.5 as coding agents in the same harness, on four tests: a two-player tank game, a real client job, a canyon flight game and a 16-bug hunt. On paper Neon should win easily. In my runs the quality was almost tied, and the time and the bill were not. If you run coding agents every day and are wondering whether to switch models, this one is for you.

What I cover in this video

  • 0:45 What Google says Neon is, and the matchup missing from its benchmark table
  • 2:18 The independent check: Artificial Analysis and Vals
  • 4:35 How I set up the bench, and why it is home field for Sonnet
  • 5:20 Event one: the two-player tank battle
  • 6:51 Event two: the client job with a changed requirement and 12 hidden tests
  • 7:36 Event three: the canyon flight, where Neon flipped the clock
  • 8:21 Event four: the bug hunt, 16 hidden bugs and 11 decoys
  • 9:52 The final scoreboard on quality, time and cost

What the charts say about Gemini 4 Neon

Google calls Neon the middle of the Gemini 4 family. Carbon is for the hardest, long jobs, Hydrogen is the fast everyday model, and Neon is supposed to balance the two. The model card says it is adapted from Carbon, with a 1 million token context window and four thinking levels. Logan Kilpatrick announced it on X and said it is punching above its model class.

In Google's own benchmark table, Neon wins 16 of 19 rows. But the table compares it with Opus 5.5 and GPT-6.2 Sol. There is no Sonnet 5.5 column, and Sonnet 5.5 is the Anthropic model in the same tier, the one a lot of us run all day. That is the matchup I ran. If the benchmark names are new to you, I explain them in Coding Agent Benchmarks Explained.

The independent boards add three things:

  • Artificial Analysis Intelligence Index: Neon scores 61, third overall. Sonnet 5.5 at its max setting scores 55.
  • Cost per task: per token, Neon costs 1.5 times what Sonnet costs. On Artificial Analysis it still comes out cheaper per task, because it reads far fewer tokens. To run the whole index, Sonnet read about 30 billion input tokens and Neon read about 8 billion.
  • Vals Index: basically a tie, 70.87% for Neon and 70.85% for Sonnet.

One more detail matters here. Those Sonnet scores are at max effort. At high effort, Artificial Analysis has Sonnet at 42 and Neon at 60, and I ran both at high. So on paper this should have been a blowout. (I tested what the effort settings change in Claude Code Effort Levels Tested.)

How I set up the test

Both models ran as coding agents inside Claude Code. Same prompt word for word, a fresh folder, effort high, web search off, no follow-ups and a time limit. Every run sat in its own sandbox, away from the hidden tests.

Fair is fair: Claude Code is Anthropic's own agent, so this is home field for Sonnet. Neon should really run in Antigravity, Google's own coding agent, but Neon was not available in Antigravity on the account on this machine. So Neon ran in Claude Code pointed at OpenRouter, with Google serving the model. Same agent for both, and the only thing swapped is the model. I did something similar with an open model in Running GLM-5.3 in Claude Code.

Neon's cost is the actual bill from OpenRouter. Sonnet ran on my own account, so I show what the same run would cost at API rates. It was one run each, so treat this as a vibe check, not a final verdict.

The four tests

Tank battle

A two-player tank game on one keyboard, built in Three.js, with three hearts each and one must-have: both players have to be able to win, with a win screen that names the winner. I played both games with the exact same key presses, once with a script where blue wins and once where red wins. Both games passed. Sonnet finished in 6.8 minutes, Neon in 12.0, and Neon's run cost more.

Client job

A dog grooming shop emails about a booking page. Halfway through, the deposit rule changes from a percentage to a flat fee, with no deposit on nail trims. There are 12 hidden tests, one per requirement, including the one that changed. Then the reply email to the client is scored blind by two judges from labs outside this matchup, DeepSeek V4 Pro and GPT-6.1 Sol.

Both models went 12 out of 12, both caught the change, and both emails got 10 out of 10 from both judges. A dead tie on quality. Sonnet took 2.1 minutes and Neon took 10.4, so Sonnet did it in about a fifth of the time, and for less.

Canyon redo

This one comes from the most-viewed Neon demo post on X: a jet, a sunset canyon, 12 gates, a timer and a saved best time. The prompts were never shared, so I wrote my own from the description. Here the clock flipped. Neon finished in 10.2 minutes and Sonnet took 22.4, for about the same money. Neon's canyon had layered red rock, a warm sunset and a light trail behind the jet. For me, Neon takes this one on looks. That is just my take.

Bug hunt

Tally House is a fake invoicing backend with 42 files where every visible test passes. I hid 16 real bugs in it, plus 11 decoys that look like bugs but are not, and nobody is told how many there are. Scoring is two points for a bug that is fixed and explained, minus one for reporting a fake bug, and minus two for breaking something that worked.

Both models fixed the same 14 of 16 and broke nothing. They even missed the same two. The judges, GLM-5.3 Prime and Mistral Large 4, both gave Neon 28 out of 32 and Sonnet 27. The one-point gap is because Sonnet explained two bugs in one entry, and the rules count one explanation per bug. Sonnet took 4.0 minutes with 16 tool calls. Neon took 17.9 minutes with 95 tool calls, and its bill was much higher.

Why the bill did not match the leaderboard

On quality, it is basically a tie. On time, Sonnet was faster on three of four: 35.3 minutes total against 50.5 for Neon. On cost, Neon's real bill was more than twice Sonnet's API-equivalent total.

Artificial Analysis says Neon is cheaper per task because it reads less. In my runs inside Claude Code, it read more: about 8.0 million input tokens across the four tests, against about 6.6 million for Sonnet. And it paid 1.5 times as much for every one of them. Meanwhile Sonnet at high effort, the 42 on the index, tied Neon's 60 on my four jobs. In Antigravity, Neon might read less and finish sooner, so the home field caveat cuts both ways.

Key takeaways

  • Gemini 4 Neon is the real deal on quality. It matched Sonnet 5.5 on every job I threw at it and took the bug hunt by one point.
  • Inside Claude Code, Neon was slower on three of four tests and cost more than twice as much overall.
  • A score at one setting does not tell you what you will get on your own work. Cheaper per task on a leaderboard can turn into more expensive in your harness.
  • For small business tools, like the shipment tracking we build in my resale business, I am keeping Sonnet. I would reach for Neon when the thing has to look good, like a landing page.
  • If you are already in Antigravity, Neon is worth a real look. This was one run each on Anthropic's home field.

FAQ

Is Gemini 4 Neon better than Claude Sonnet 5.5 for coding?

In my four tests inside Claude Code, quality was basically a tie. Both passed the tank game, both went 12 out of 12 on the client job, and both fixed the same 14 of 16 bugs, with Neon ahead 28 to 27 out of 32 on the bug hunt. Sonnet was faster on three of four tests.

Is Gemini 4 Neon cheaper than Claude Sonnet 5.5?

Per token, Neon costs 1.5 times what Sonnet costs. On Artificial Analysis it comes out cheaper per task because it reads fewer tokens, but in my runs inside Claude Code it read more input tokens than Sonnet (about 8.0 million versus 6.6 million) and its total bill was more than twice Sonnet's API-equivalent cost.

Can you run Gemini 4 Neon in Claude Code?

Yes. I ran Neon in Claude Code pointed at OpenRouter, with Google serving the model. Claude Code is Anthropic's own agent, so that is home field for Sonnet, and Neon might read less and finish sooner in Antigravity.

How do Gemini 4 Neon and Sonnet 5.5 compare on independent benchmarks?

On the Artificial Analysis Intelligence Index, Neon scores 61 and Sonnet 5.5 at max effort scores 55, with Sonnet at 42 at high effort. On the Vals Index it is basically a tie, 70.87% for Neon and 70.85% for Sonnet.

Run your agents in Beckon

A workspace for your coding agents. Get more work done in less time.

Download BeckonSubscribe on YouTube
Full video transcript

0:00Google just dropped Gemini 4 Neon. On artificial analysis, it scores a 61. Sonnet 5.5 scores a 55. And Neon costs less per task. On paper, game over. So, we ran both in the same coding agent on the same four tests. The scoreboard came back almost tied. The bill didn't. Neon cost more than twice as much. So, in this video, a tank battle, a real client job, a canyon flight from the most viewed neon post on X, and a bug hunt with 16 hidden bugs. Then, the scoreboard with real time and real cost. I run coding agents every day. So, this is what I care about. Let's jump right into it. So, what is Neon exactly?

0:45Google calls it the middle of the Gemini 4 family. Carbon is for the hardest, long jobs. Hydrogen is the fast everyday one. and Neon is supposed to balance the two. Google's own model card says it's adapted from carbon. A 1 million token context window, four thinking levels, and it's the fourth Gemini 4 model in about 7 weeks. Logan Kilpatrick announced it on X, and that post is past 370,000 views. In his words, it's punching above its model class. You can get it on the paid tier of Google's API in AI studio and in anti-gravity on the paid pro and ultra plans and it's on open router which matters for our bench in a minute. Now Google's own benchmark table. Neon

1:30wins 16 of 19 rows. But look at who's in the table. Opus 5.5 and GPT 6.2 Soul. There's no Sonnet column. Sonnet 5.5 is Anthropic's model in the same tier, the one a lot of us actually run all day. And that's the matchup Google didn't print. So that's the one we're running. And the two rows Opus still wins, Terminal Bench and Frontier Code, the hard coding agent rows. Google's numbers. So take that for what you will. One more thing. Per token, Neon costs one and a half times what Sonic costs. Hold on to that because it comes back. Real quick, every video on this channel is made in code with Claude code. Inside Shipping School Premium, I show you how to turn your own product into videos that get views step by step, plus a live

2:18Q&A with me every Saturday. Join free link below. So, Google's table is Google grading Google. What do the independent board say? Artificial analysis has Neon at a 61 on their intelligence index, third overall behind Fable 5.2 and GPT 6.2 Astra. Sonnet 5.5 at its max setting is a 55. Now, this is the part I didn't expect. Neon costs more per token, but on artificial analysis, it's cheaper per task. About $6.40 a task versus about $10 for Sonnet. So, how does the pricier model come out cheaper? It reads way less. To run that whole index, Sonnet read about 30 billion input tokens. Neon read about eight. You pay per token, so

3:05fewer tokens means a smaller bill. Val is the other big independent board. And over there, it's basically a tie. 70.87% for Neon, 70.85 for Sonnet. And Sonnet still wins one on Artificial Analysis's office work test called Briefcase. Sonnet's ahead 1815 to,700. Okay, so this is the sneaky part. Those sonnet scores are at max effort where it thinks a long time. At high effort, artificial analysis has sonnet at a 42. Neon at high is a 60. And we ran both at high. So on paper, this should be a blowout. So what did people build on day one? The most viewed neon demo post on X is from Kimismus. About 48,000 views. He

3:50had early access and built 17 demos in anti-gravity, Google's own coding agent, a neon canyon rail shooter, a voxil castle you can blow apart, where Neon added tools he never even asked for, and a jet flying through 12 gates in a sunset canyon. Remember that one? Then people started comparing. Not Jazzy put it up against Opus 5.5, and he said Neon added crazy detail, but in his words, Opus still did way better. Supernew ran a rainy night convenience store against GPT 6.2 Soul and Neons came out dim and empty. The same guy also posted a Pelican test, and that one he gave to Neon. And Kimismas logged a bunch of no capacity errors during early access,

4:35which he says could just be pre-launch load. So, it's the same story as every launch. A great chart and then real people start using it. So, here's how we ran it. Both models ran as coding agents inside Claude code. Same prompt, word for word, a fresh folder, high effort, web search off, no follow-ups, and a time limit. Every run sat in its own sandbox away from the hidden tests. Now, fair is fair. Claude Code is Anthropic's own agent, so that's home field for Sonnet. Neon should really run in anti-gravity. But Neon in anti-gravity needs a paid Google plan and the free account on this machine only offers flash. So Neon ran in claude code pointed at open router with Google

5:20serving the model. Same agent for both and the only thing we swap is the model. Every cost on screen comes from the actual run for Neon. That's the actual bill from open router. Sonnet ran on my subscription. So we show what the same run would cost at API prices. One run each. So, it's a vibe check, not a final verdict. Event one, the game. A two-player tank battle on one keyboard built in 3.js. Blue on the left, red on the right, three hearts each, and one mustave. Both players have to be able to win with a big win screen that names the winner. We played both games with the exact same key presses. Sonnet finished in under 7 minutes, about 97 cents at API prices. yellow hazard stripes around

6:05the arena, crates for cover, and a score counter up top. Neon took 12 minutes, and the bill was about $2. A clean gray arena, aimlines on the floor, and framed hearts for each player. Which one looks better to you? Then the must-have. We ran one script where blue wins and one where red wins. Both games showed blue wins. Both games showed red wins, so both pass. Honestly, I'd play either one with my kids. Event two is the kind of work I'd actually pay someone for. A dog grooming shop emails about a booking page. Halfway through, one of them changes the deposit rule from a percentage to a flat fee with no deposit at all on nail trims. And they want bookings not more than a couple months out, which is up to you to read. There

6:51are 12 hidden tests, one per requirement, including the one that changed. Then the email back to the client scored blind by two judges from labs that aren't in this matchup. Deepseek V4 Pro and GPT 6.1 Soul. Sonnet 2 minutes, 12 out of 12. Neon 10 1/2 minutes, 12 out of 12. Both caught the change and both emails got a 10 out of 10 from both judges. So it's a dead tie on quality. But Sonnet did it in a fifth of the time for 39 cents at API prices. Neon's bill was $1.69. So far, sonnets's ahead on the clock. And then the canyon flipped it. Event three comes from that most viewed Neon thread, Kimonismos's

7:36Arrow Canyon. He never shared his prompts, so we wrote our own from what he described. A jet, a sunset canyon, 12 gates, a timer, and a saved best time. Neon finished in 10 minutes for $161. Sonnet took 22 minutes, $164 at API prices, so about the same money. And Neon finished in less than half the time. And look at Neon's Canyon. Layered red rock, a warm sunset, and a light trail behind the jet. Sonnets has the gates lit up in blue, and a crash screen every time you hit a wall. With my blind key presses, that was a lot. Which one looks better to you? For me, Neon takes this one on looks. That's just my take. Event four is the hardest one, the bug

8:21hunt. Tally House is a fake invoicing backend. 42 files and every visible test passes. But we hid 16 real bugs in it. The kind that cost a business real money. A double charge, a refund booked in the wrong month, plus 11 decoys that look like bugs but aren't. And nobody's told how many there are. Think of it as a money leak meter. 16 leaks and every fix that passes its hidden test plugs one. Two points for a bug that's fixed and explained, minus one for reporting a fake bug, and minus two for breaking something that worked. Sonnet took 4 minutes. 14 of 16 plugged, nothing broken. Neon took 18 minutes. 14 of 16, nothing broken. The same 14. They even

9:07missed the same two. a cash that fills right as it's being cleared and a refund that should turn the payment reminders back on. The judges, GLM 5.3 Prime and Mistral Large 4, both gave Neon a 28 out of 32 and Sonnet a 27. Why one point? Sonnet explained two bugs in one entry, and the rules count one explanation per bug. That's the whole gap. Neon made 95 tool calls. Sonnet made 16. Neon's bill was $3.14. Sonnetss was 89 cents at API prices. So here's the final scoreboard on quality. It's basically a tie. Both games play and both pass the must-have. Both client jobs 12 for 12. The bug

9:52hunt, Neon by one point. And the Canyon Neon on looks in my book. On time, Sonnet was faster on three of four. 35 minutes total against 50.5 for Neon. And on cost, Neon's real bill was $8.50. 50. Sonnet 3.89 at API prices. More than twice as much for Neon. So remember that per token versus per task thing. Artificial analysis says Neon is cheaper per task because it reads less. In our runs inside Claude Code, it read more. About 8 million input tokens for Neon across the four tests and about 6.5 million for Sonnet and it paid one and a half times the price for every one of them. and sonnet at high effort the 42 on our four jobs it tied neon 60 so a

10:40score at one setting doesn't tell you what you'll get on your own work fair note that was anthropic's home field in anti-gravity neon might read less and finish sooner so the four questions can it make a game yes neon's tank battle plays both players can win and to me its canyon was the best looking thing in this whole test can it hunt bugs 14 of 16. Nothing broken and the top score by a point. Same bugs as Sonnet though in more than four times the time. What does it mean for AI coders? Neon is the real deal. It matched Sonnet on every job we threw at it. But if you live in cloud code like I do, it's slower and it costs more in there. If you're already in anti-gravity, it's worth a real look. And how would I use it in a business? In

11:27my resale business, we build little tools all the time like tracking for our shipments. For that, I'm keeping Sonnet. It did the client job in two minutes for under 40 cents at API prices. I'd reach for Neon when the thing has to look good, like a landing page. So, where do you land? Is Neon worth leaving Sonnet for, or does it need Google's own agent to really shine? Tell me in the comments. And if you want the rematch inside Anti-Gravity, hit like and I'll run it. Have a blessed day.