In this video I fill Claude Code's context window to about 840,000 tokens with a real earlier work session, then give Claude Opus 5.5 and Claude Sonnet 5.5 four jobs: a game, a client job, a 3D build and our hardest bug hunt. A lot of people say your agent gets dumber at 500K. In our test it didn't, but the token count, the API-equivalent cost and the room left to work all changed a lot. If you keep one Claude Code session open all afternoon, this one is for you.
What I cover in this video
- 0:46 What a context window is and what fills it, starting from about 28K tokens before the agent types a word.
- 2:17 What the research says about context rot, and where the published tests stop.
- 3:47 The setup: Opus 5.5 vs Sonnet 5.5 at four fill levels, filled with a real Fastify session.
- 4:32 Test one, a Subway Surfers style game in one file, and the first time we hit the wall.
- 5:20 Test two, a messy client email thread with 12 hidden tests.
- 7:39 Test four, the bug hunt: 3,000 lines, 16 hidden bugs, plus decoys.
- 8:25 Everything on one chart: flat quality, about 14 times the tokens, and where the money goes.
- 9:55 What to do about it: fresh sessions, compaction and sub-agents.
What fills the context window
The context window is everything the model can see at once: your messages, its answers, and every single thing its tools bring back. In our runs, Claude Code was already sitting at about 28,000 tokens before the agent typed a word. That is its system prompt, the tool definitions and some setup info.
Then the session starts, and every file read, grep result, test log and git diff stays in there. Anthropic's docs walk through an example where one source file is about 1,000 to 2,500 tokens and one npm test run is about 1,200. One noisy command can dump up to 30,000 characters of output into the window. In our test, code and logs came out to about two and a quarter characters per token, so Fastify's reply file alone is about 14,000 tokens.
The part that matters for cost: Claude Code sends your full conversation with every request, and every tool call is another request carrying the whole history again. Opus 5.5 and Sonnet 5.5 both have a 1 million token window, and Claude Code doesn't auto-compact them until about 967,000 tokens by default. So a 500K session isn't an edge case. It's a normal afternoon in one session.
What the research says, and how I set up the test
People call this context rot, and Anthropic uses the term in its own docs: as token count grows, accuracy and recall degrade. The big study is from Chroma. They tested 18 models and found performance consistently drops as the input gets longer, even on simple tasks. But those were earlier models, and their most realistic test stopped at about 113,000 tokens. Artificial Analysis runs a long context test at about 100,000 tokens, where Opus 5.5 scored 84.7 at max effort and Sonnet 5.5 scored 82.7. Anthropic reports its own long context coding number up to the full million: 91.2 for Opus and 79.7 for Sonnet. What I couldn't find anywhere was these two models measured at 500K and up, so we did it ourselves.
Both models ran in Claude Code with effort high, web search off, sandboxed, at four fill levels: fresh (about 28K), 100K, 500K and 850K, which really landed around 840K. We didn't fill the window with lorem ipsum. We built a real earlier session where the agent spent the morning upgrading an API to the latest Fastify: reading the source, docs and tests, running git log and the real test suite, 392 steps of real output. Then we resumed that session and handed it a brand new job.
Auto-compact was off for every run so the window stayed as full as we set it. Every run got the same prompt, hidden tests the agents never see, and two blind judges from other labs, Kimi K3.1 Pro and GLM-5.3 Prime. That's 28 runs in all.
The results: quality stayed flat
- The game. All four games play, and I honestly can't tell the 850K ones apart from the fresh ones. Opus took 24 minutes fresh and 20 at 850K. Sonnet took 8 minutes fresh and 17 at 850K.
- The client job. Sonnet passed 12 of 12 hidden tests at every level. Opus got 12 fresh and at 500K, and 11 at 100K and 850K. The miss was a reminder email that Opus put in a text box our test couldn't read. With a test that reads text boxes, it's 12 of 12. Every run caught the mid-thread tax change, and every run asked about the undefined late fee instead of making one up.
- The galleon. Three of four ships work. Opus at 850K looks just as good as the fresh ones. Sonnet's 850K ship never loads, and that failure was the wall cutting it off, which I get to below.
- The bug hunt. Opus fixed 14 of 16 bugs fresh, at 100K, at 500K and at 850K. On second runs it fixed 15 fresh and 14 at 850K. Sonnet fixed 13, 15, 15 and 14. Zero false reports and zero regressions.
Where it breaks: tokens, cost and the wall
Opus's bug hunt processed 1.86 million tokens fresh and 25.37 million at 850K. That's about 14 times more for the exact same 14 bugs, and about 6.6 times the API-equivalent cost. Nothing was actually billed for these runs, but that's what the same work would come to at API rates. Most of that cost is loading the old session into the cache once, then reading it again on every request: 28 requests, each carrying about 840,000 tokens or more. I look at caching on its own in this caching test.
Time went up too, but not in a straight line. Sonnet's bug hunt took under 4 minutes fresh and 7 to 9 minutes at 850K.
Then there's the wall. Four of our 10 runs that started at 850K hit Claude Code's limit just under 1 million tokens and stopped with "Prompt is too long". Sonnet's game was already saved, so it still plays. Its galleon wasn't so lucky: the last edit broke the file, and the session ended before it could test it. On the bug hunt, Sonnet's first 850K run fixed 14 bugs in the code and then hit the wall before writing its report, so the judges had nothing to read and it scored 14 points out of 32. The second time it finished and scored 30 out of 32.
In a normal session with auto-compact on, Claude Code would have summarized the session at about 967K, right in the middle of the task, and a summary can lose details. So the model held up fine. It just ran out of room.
What to do about it
- Start a fresh session for each job. Clearing costs nothing, and every request after that starts from about 28K instead of 850K.
/clear
- Use compaction on purpose. It summarizes the conversation, keeps your requests and key snippets, and rereads up to five recent files. It is not free and early details can get lost (more in what compaction keeps and forgets). You can move the trigger, say to 500K, with:
/autocompact
- Send heavy reading to sub-agents. Each one starts with its own fresh window, does the reading over there, and sends back a short summary, so your main window stays small. Anthropic says it straight: handing a job to a sub-agent doesn't make the answer any better. I tested that in sub-agents vs one agent.
Key takeaways
- A full window didn't make Opus 5.5 or Sonnet 5.5 dumber in our test. Bug hunt results were 13 to 15 of 16 at every fill level, the same as fresh.
- What changed was everything around the model: about 14 times the tokens and about 6.6 times the API-equivalent cost on the Opus bug hunt.
- Four of 10 runs that started at 850K hit the limit, and the damage depends on whether the agent had saved and checked its work yet.
- Start each job fresh and send anything that reads a ton of files to a sub-agent. You burn a fraction of the tokens for the same work.
- This is a small sample: one run per cell, with a second bug hunt run at fresh and 850K. Take it for what it is.
Full video transcript
0:00I filled Claude Code's context window to about 840,000 tokens with old work, then handed Opus the hardest bug hunt we have. It fixed the same 14 bugs it fixed in a brand new session. A lot of people say your agent gets dumber at 500K. In our test, it didn't. It just costs more than six times as much at API prices, and four of our runs ran out of room. In this video, what fills the window? Four fill levels, four questions. Can it make a game? Can it hunt bugs? What does it mean for AI coders and for your business? I run coding agents all day, so I wanted real numbers. Let's jump right into it. So, what is a context window? It's everything the model can see at once. Your messages, its answers,
0:46and every single thing its tools bring back. And it fills up faster than you think. In our runs, before the agent typed a word, Claude Code was already sitting at about 28,000 tokens. That's its system prompt, the tool definitions, and some setup info. Then the session starts. Every file it reads stays in there. Every GP result, every test log, every git diff. Anthropic walks through an example in their docs. One source file is about 1 to 2 1/2 thousand tokens. One npm test run about 1,200. Now do that all afternoon. And one noisy command can dump up to 30,000 characters of output straight into the window. In our test, code and logs came out to about two and a quarter characters per
1:31token. So one file, Fastifi's reply file, is about 14,000 tokens on its own. Now flip it around. Claude Code sends your full conversation with every request. Their cost page says that word for word. Every time the agent uses a tool, that's another request carrying the whole history again. And Opus 5.5 and Sonnet 5.5 both have a 1 million token window. Claude Code doesn't autoco compact them until about 967,000 tokens. So a 500k session isn't some edge case. If you work in one session all afternoon, that's just a normal Tuesday. So here's what I wanted to know. When that window is mostly full, does the agent actually get worse at the job? And the one almost nobody asks,
2:17"What does all that old work cost you on every single step?" Real quick, every video on this channel is made in code with Claude Code. Inside Shipping School Premium, I show you how to turn your own product into videos that get views step by step, plus a live Q&A with me every Saturday. Join free link below. So, people have a name for this, context rot, and Anthropic uses it in their own docs. As token count grows, accuracy and recall degrade. The big study is from Chroma. They tested 18 models and found performance consistently drops as the input gets longer, even on really simple tasks. In their most realistic test, every model did better with a short focused prompt than with the full chat
3:02history of about 113,000 tokens. But those were earlier models, the Claude 4 and GPT 4.1 generation, and that realistic test stopped at about 113,000 tokens. Artificial analysis runs a long context test, too, at about 100,000 tokens. Opus 5.5 scored 84.7 at max effort. Sonnet 5.5 82.7. Anthropic's own long context coding number goes up to the full million, and they report 91.2 2 for Opus and 79.7 for Sonnet. Those are their numbers. And what I couldn't find anywhere was these two models measured at 500K and up. So, we did it ourselves. Here's the matchup. Opus 5.5 versus
3:47Sonnet 5.5. Both in Claude Code, effort high, web search off, sandboxed, and each one plays at four fill levels. fresh, which is about 28K, 100K, 500K, and 850K, which really landed right around 840. So, how do you fill a context window with real work? We didn't use Lauram Ipsum. We built a real earlier session. The story is you've been upgrading your API to the latest Fastify all morning. So, the agent has read Fastifi's source, its docs, and its tests, run git log, and run the real test suite. 392 steps of real output. Then we resume that session and hand it a brand new job. Why 850 and not a
4:32million? Because the agent needs room to work. Start at a million and it can't even read the first file. We also turned auto compact off for every run so the window stays as full as we set it. Most people leave it on and that matters later. Same prompt for every run. Hidden tests the agents never see. And two judges from other labs. Kimmy K 3.1 Pro and GLM 5.3 Prime scoring blind. 28 runs in all. Test one. The game Subway Surfers in 3JS one file fresh versus 850 for both models and every game gets the exact same key presses. So you can see all four of them run lanes, jumps, coins, the crash screen. Opus fresh took 24 minutes. Opus at 850 took 20. Sonnet
5:20fresh 8 minutes. Sonnet at 850 17 and this is the first time we saw the wall. Sonnet at 850 saved its game, kept working on it, and then the session just stopped. Prompt is too long. It hit the limit. The game was already saved, so it still plays, but if it hadn't saved yet, you'd have nothing. Which one looks better to you? I honestly can't tell the 850 ones apart from the fresh ones. Test two is the one I care about for real work. A messy email thread from a client named Dana. Dana wants an invoice tracker. Then halfway through, Dana's accountant changes the tax rule. Tax only on printing, not on design. And there's a late fee nobody actually pinned down. 12 hidden tests, one per requirement. Sonnet passed 12 out of 12
6:06at every single level. Fresh 100k, 500k, 850. Opus got 12 fresh and at 500k and 11 at 100k and at 850. Okay, so this is the sneaky part. The one it missed was the reminder email. Opus put the email in a text box and our test reads the pages text which comes back empty for a text box. We reran it with a test that reads text boxes and it's 12 out of 12. You can see the email right there in the recording. So officially 11, but the app works. Every run caught the tax change. Every run asked about the late fee instead of making one up. That's not an agent that got dumber. Test three. This gallion was making the rounds on X. Marcel at Mark the Creator built it with
6:53Sonnet 5.5 and put it next to Gemini 4 hydrogen. Almost 16,000 views. He never shared the prompt, so we wrote our own. An ornate golden gallion. Moving water day and night. Three camera modes. Look at these. Opus at 850 looks just as good as the fresh ones. Opus Fresh did need a restart. Nothing saved after 15 minutes. And our harness restarts any build like that once. Same rule for everybody. And the wall again. Both 850 runs hit the limit. Opus had already saved a working ship. Sonnet's last edit broke its file and it hit the limit before it could test it. So Sonnet's 850 ship never loads. Just a black screen. The wall cut it off before it could check its own work. Test four. The hard one. Tally
7:39house. An invoicing backend. 3,000 lines, 16 hidden bugs, and the agent is never told how many, plus decoys that look like bugs but aren't. Two points for a bug fixed and explained, minus points for false reports. Opus fixed 14 fresh, 14 at 100K, 14 at 500, 14 at 850. We ran fresh and 850 a second time, and it fixed 15 and 14. Every bug Opus missed was one of the same two hard ones at every level. Zero false reports, zero things broken. Sonnet fixed 13 fresh, 15 at 100K, 15 at 500, and 14 at 850. But that 850 run only scored 14 points out of 32. And it didn't miss bugs. It fixed
8:2514 of them in the code, and then it hit the wall before it could write the report. So the judges had nothing to read. The second time at 850, it finished and scored 30 out of 32, tied for its best score all day. So, let's put it all on one chart. The quality line is flat. Fresh to 850. Same bugs, same client job, same games. Now, the other line. Opus' bug hunt read 1.9 million tokens fresh. At 850, 25 million. That's about 14 times more for the exact same 14 bugs. The API equivalent cost went from $2 to 1329. Sonnet went from 79 to 638. These ran on my subscription, so nobody got build, but that's what it would cost on the
9:10API. And here's where the money goes. About $75 of that 13 is loading the old session into the cache one time. Most of the rest is reading it again on every request. 28 requests, each one carrying about 840,000 tokens or more. Time went up, too, but not in a straight line. Sonnet's bug hunt took under 4 minutes fresh and 7 to 9 minutes at 8:50. And then the wall. Four of our 10 runs that started at 850 hit the limit. In a normal session with autocompact on, Claude Code would have stopped at about 967 and summarized your session right in the middle of the task. And a summary can lose details. So the model held up fine. It just ran out of room. So what
9:55do you actually do with this? Three tools. And here's how each one works. One, a fresh session. Clear costs nothing. Every request after that starts from about 28K instead of 850. Anthropics docs say it too. A focused context gets you better answers and costs less. Two, compaction. It summarizes the conversation, keeps your requests and key snippets, and rereads up to five recent files. It is not free and early details can get lost. You can also move the trigger with /auto compact, say to 500K. Three sub aents. Each one starts with its own fresh empty window, does the heavy reading over there, and only sends back a short summary, so your main window stays small. Just know, Anthropic says it
10:41straight. Handing a job to a sub agent doesn't make the answer any better. In my resale business, we're analyzing a ton of sales data with agents. After this, I'd start a fresh session for every new job, and anything that reads a ton of files goes to a sub agent. So, back to the four questions. Can it make a game at 850? Yes. All four played and they look just as good. Can it hunt bugs? Yes. 13 to 15 out of 16 at every level, the same as fresh. What does it mean for AI coders? A full window didn't make these models dumber in our test. It made them slower on most jobs, a lot more expensive at API prices, and four times they hit the wall. And for your business, the client job came out the same at every level once you count that
11:26text box. Start each job fresh and you burn a fraction of the tokens for the same work. Small sample, one run each, plus a second bug hunt run at fresh and 850. So, take it for what it is, but I'll be clearing my sessions a lot more now. And if you'll be watching that context meter a little closer now, hit like. It helps this reach more people running long sessions. So, how full do you let yours get before you clear it? Tell me down below. Have a blessed day.