In this video I run the exact same Claude Code job twice, once with prompt caching on and once with it off, and compare the bills. If you run coding agents for long sessions, this one is for you: caching is the reason an agent running all day doesn't cost a fortune, and you can break it without noticing. I explain why agents resend everything, how a cache hit works, what breaks it, and what I do to keep my hit rate high.
What I cover in this video
- 0:00 Same job, same model, same bugs, two very different bills
- 0:46 Why your agent resends the whole conversation on every call
- 2:19 How a cache hit actually works, and why order matters
- 3:50 Cache markers, automatic caching and the 20 block lookback
- 4:35 Cache lifetime in Claude Code, and the things that break it
- 6:07 Our test run, with caching versus without
- 8:25 How to keep your cache hits high
Why your agent resends everything
The API has no memory. Anthropic's docs for the Messages API say it plainly: it's stateless, so every request has to carry the whole conversation from the top.
Here's the loop. The agent sends a request, the model answers with a tool call (like "read this file"), Claude Code runs it on your machine, and then it sends a brand new request containing everything from before, plus the model's answer, plus the file it just read. Every tool call is a full round trip. (If you want a deeper walkthrough of that loop, see How the Claude Code Agent Loop Works, One Tool Call at a Time.)
You can see it in our run. On the very first call, before the agent read a single file, Claude Code was already sending 28,026 tokens: its system prompt, its tool definitions and our prompt. By the end, the uncached run was sending 121,934 tokens on a single call, just to take one more step. Manus, the agent company, wrote that their agents take in about 100 tokens for every one they write. So for an agent, almost the whole bill is input, and almost all of that input is stuff the model saw a few seconds ago.
How a cache hit works, and what breaks it
When a model reads your prompt, it does a lot of computation on every token before it writes anything back. Prompt caching saves that work for the start of the prompt (the prefix), so the next request that starts with the exact same tokens skips past it.
Order matters. Anthropic lays out the prompt as tools first, then the system prompt, then the messages, and the match is exact from the very first token. If one character changes, everything after that point is a miss. So every call has three kinds of input:
- Cache read: the part that matched
- Cache write: the new part being saved for next time
- Plain input: anything after the last save point
Per Anthropic's pricing page, writing to the cache costs a bit more than normal input, but on Sonnet 5.5 and Opus 5.5 a cache hit costs 5% of normal input. On most other Claude models reads are 10%, and even there the 5 minute cache pays for itself after a single read.
To tell the API where to save, you add a cache_control marker (up to four of them), or turn on automatic caching, which moves the marker forward as the conversation grows. On each request it looks back up to 20 blocks for a match, and a batch of parallel tool calls counts as just one of those 20.
The clock
By default the cache lasts 5 minutes, and every hit resets the timer for free. There's also a 1 hour option that costs more to write. The clock starts when the request starts, not when the answer finishes, so if a response streams for 4 minutes you have about 1 minute left to get the next hit.
According to the Claude Code caching docs, on a subscription (within your usage) the main conversation uses the 1 hour cache. On an API key it's 5 minutes unless you turn on a setting, and sub-agents get 5 minutes by default either way. The Claude Code cost docs note that if you step away longer than the cache lasts, your first message back reprocesses the entire context. Go to lunch, type "keep going", and you pay for the whole conversation again.
What breaks the cache
- Changing tool definitions wipes the whole cache.
- Adding or removing an image, changing tool choice, or changing thinking settings wipes the messages part.
- Switching models mid-session means the next request gets no hits at all. The Claude Code team pointed out that 100,000 tokens into an Opus chat, switching to Haiku for a quick question costs more than staying on Opus, because Haiku has to read all 100,000 tokens from scratch. I switch models all the time, so I'm guilty of this one.
Our run: caching on versus off
I wanted a real job, not a docs example. The setup: a small invoicing app with 42 files, 16 planted bugs, and hidden tests that check every fix. Claude Sonnet 5.5 at high effort in Claude Code, same prompt word for word. One run used the default (caching on). The other set a single environment variable:
DISABLE_PROMPT_CACHING=1
We started both at the same time, then repeated the whole thing for a second round.
All four runs fixed the same 14 of 16 bugs, and none broke a single test. The work was identical. The bill wasn't: on average, turning caching off cost about 4.2x as much at Anthropic's list prices. I ran this on my subscription, so these are API-equivalent numbers for the same tokens.
The uncached run in round two took a few more steps, which pads that ratio a little. So the fairest comparison: take the cached runs' exact tokens and price them as if there were no cache. Caching took about 75% off the exact same traffic.
The per-call numbers show why. On call 2, about 28,000 tokens came back as a cache hit and only about 2,500 new tokens were written. By call 10, it read 91,000 tokens from cache and wrote 577 new ones. Over 90% of input tokens were cache hits in both rounds. Without the cache, every call pays full price for the whole stack.
The cached runs also finished a bit sooner both times, but with two rounds and different paths I won't promise a speed number. Anthropic's docs say faster time to first token, and on the API, cache reads don't count toward your input tokens per minute limit, so a high hit rate lets you push more through before throttling.
How to keep your cache hit rate high
The Claude Code team said they watch cache hit rate like uptime, and treat a drop as an incident. Manus called it the single most important metric for a production agent. Here's what they do:
- Stable stuff first, changing stuff last. Tools and instructions at the top, new messages at the bottom.
- No timestamp at the top. One changing token at the start and nothing after it matches.
- Append only. Don't edit what's already in context, and keep JSON keys in the same order, since a different order is a different prefix.
- Don't add or remove tools mid-session. Manus hides tools instead of removing them, and Claude Code puts updates into the messages rather than rewriting the system prompt.
For you in Claude Code: pick your model up front and stick with it, try not to walk away in the middle of a big task, and if you're on an API key and take long breaks, look at the 1 hour cache setting. Also, tiny prompts don't cache at all. On Sonnet and Opus 5.5 the minimum is 512 tokens, and anything shorter skips the cache with no error.
Key takeaways
- The Messages API is stateless, so agents resend the full conversation on every tool call, and input dominates the bill.
- Caching matches an exact prefix in order: tools, system prompt, messages. One changed character breaks everything after it.
- In our test, caching off cost about 4.2x more for identical results (14 of 16 bugs fixed in all four runs).
- Stepping away past the cache lifetime, switching models, or changing tools or thinking settings forces a full reprocess.
- Keep stable content first, append only, and choose your model before you start.
Full video transcript
0:00We ran the exact same claude code job with prompt caching on and then with it off. Same model, same prompt, same bugs. With caching on, it cost 79 at API prices. With it off, 2.77. And both runs fix the exact same 14 bugs. Most people think caching is some back-end thing the provider handles for you. It's not. It's the reason running an agent all day doesn't cost a fortune, and you can break it without ever knowing. So, in this video, why your agent rescends everything, how a cash hit works, what breaks it, our runs side by side, and how to keep your hits high. I run these agents every day, so I wanted real numbers. Let's jump right into it. So, why would an agent send the same stuff
0:46over and over again? Because the API has no memory. Anthropic docs say it straight. The messages API is stateless. Every request has to carry the whole conversation from the top. Here's how it actually works. Your agent sends a request. The model answers with a tool call like read this file. Cloud code runs it on your machine and then it sends a brand new request with everything from before plus the model's answer plus the file it just read. And then it does that again and again. Every single tool call is a full round trip. You can see it in our run. On the very first call, before the agent read a single file, Claude Code was already sending 28,000 tokens. That's its system prompt, its tool definitions, and our
1:33prompt. By the end of the job, the run without caching was sending over 120,000 tokens on a single call. And it sends that whole thing every time it wants to take one more step. Manis, the agent company, wrote that their agents take in about a 100 tokens for every one they write. 100 to one. So for an agent, almost the whole bill is input and almost all of that input is stuff the model already saw a few seconds ago. So the API doesn't remember your conversation, but it can remember the work it already did on the start of your prompt. And that's prompt caching. And the discount is a lot bigger than you'd think. Want to make videos like this one with Claude Code? That's what I teach step by step inside Shipping School Premium. New lessons every single day,
2:19live calls every week, a full course library, and over 500 hours of replays. 263 builders are already in. Join free link below. So, how does a cache hit actually work? When a model reads your prompt, it does a ton of math on every token before it can write a single word back. Prompt caching saves that work for the start of the prompt, the prefix, so the next request that starts with the exact same tokens can skip right past it. The order matters a lot. Anthropic lays out the prompt as tools first, then the system prompt, then the messages and the match is exact from the very first token. If one character changes somewhere, everything after that point is a miss. So on every call, there are three kinds of input. A cache read,
3:05that's the part that matched, a cache write, that's the new part getting saved for next time, and plane input, which is anything after the last save point. Now, the prices, these are from Anthropic's own pricing page as of November 9th. Sonnet 5.5 is $2 per million input tokens. Writing to the cash costs a little more, $250 for the 5minute cache or $4 for the 1 hour cash, but reading from the cash is 10 on Sonnet and Opus 5.5. A cash hit costs 5% of normal input, which is insane. So, you pay a little extra once to save it, and every hit after that is pennies on the dollar. On most other cloud models, reads are 10%. And even there, the 5-minute cache pays for itself after just one read. So,
3:50how does it know where to save? You put a marker in the request called cache control, and you get up to four of them. Or you turn on automatic caching, and it moves the marker forward for you as the conversation grows. Then, on each request, it looks back up to 20 blocks to find a match. And a batch of parallel tool calls only counts as one of those 20. So, a busy turn doesn't knock your last save out of the window, but all of this has a clock on it. Now, the fine print. By default, the cash lasts 5 minutes, and every time you hit it, the timer resets for free. There's also a 1 hour option that costs more to write. And the clock starts when the request starts, not when the answer finishes. So, if a response takes 4 minutes to stream, you've got about 1 minute left
4:35to send the next one and still get the hit. Claude Code handles a lot of this for you. On a subscription, as long as you're inside your plan's usage, the main conversation uses the 1-hour cache. On an API key, it's 5 minutes unless you turn on a setting. And sub agents get 5 minutes by default either way. So, this is the part that bugged me. Claude Code's own cost docs say if you step away longer than the cache lasts, your first message back reprocesses the whole context. So, you go grab lunch, you come back, you type keep going, and you pay to process the entire conversation all over again. Come on now. And the clock isn't the only thing that breaks it. Change your tool definitions and the whole cache is gone. Add or remove an image, change tool choice, or change
5:22your thinking settings, and the messages part is gone. Switch models in the middle of a session, and the next request gets no cash hits at all. The Claude Code team even said that a 100,000 tokens into an Opus chat, switching to Haiku for a quick question costs more than just staying on Opus because Haiku has to read all 100,000 tokens from scratch. And I switch models all the time, so I'm guilty of this one. So, how much does all of this actually save on a real job? We tested it. So, I wanted to see this on a real job, not a docs example. One bug hunt. A small invoicing app. 42 files, 16 bugs planted in it, and hidden tests that check every fix. Claude Sonnet 5.5. High effort in
6:07Claude Code. Same prompt word for word. One run with caching on, which is just the default, and one run with a single environment variable, disable prompt caching, set to one. We started both at the same time, and then we did the whole thing again for a second round. So, what happened? All four runs fixed the same 14 bugs out of 16 and none of them broke a single test. The work was identical. The bill wasn't. Round one, cashing on cost 79. Cashing off 2.77. Round two, 78 versus 3.83. So on average, turning cashing off cost about 4.2 times as much. That's at Anthropic's list prices. I ran this on my subscription. So this
6:53is what those same tokens would cost you on the API. Now the offun in round two took a few more steps which pads that number a little. So here's the fairest way to look at it. Take the cash runs exact tokens and price them like there was no cash 309. It actually cost 79. That's 75% off on the exact same traffic. And you can see why when you look at each call. On call two, 28,000 tokens came back as a cash hit and only about 2500 new tokens got written. By call 10, it read 91,000 tokens from the cache and wrote 577 new ones. Over 90% of everything it sent was a cash hit both rounds. Without the cash, every one of those calls pays full price for the
7:39whole stack. Call 15 of the offrun sent almost 122,000 tokens at full price just to take one more step. The cached runs also finished a little sooner both times, but that's two rounds and the runs took different paths, so I'm not going to promise you a speed number. Anthropic docs just say faster time to first token. And if you're on the API, cache reads don't count toward your input tokens per minute limit. So with a high hit rate, you can push a lot more through before you get throttled. So, how do you keep that hit rate up? The Claude Code team said they watch cash hit rate like uptime. If it drops too low, they treat it as an incident. And Manis called it the single most important metric for a production agent. So, here's what they do. One, stable
8:25stuff first, changing stuff last. Your tools and instructions at the top, the new messages at the bottom. Two, don't put a timestamp or anything that changes at the very top of your prompt. Manis called that one out by name. one changing token at the top and nothing after it matches. Three, only add to the context. Don't edit what's already there. And keep your JSON in the same order every time because a different key order is a different prefix. Four, don't add and remove tools in the middle of a session. Manis hides tools instead of removing them, and Claude Code puts updates into the messages instead of rewriting the system prompt. And then for you in Claude Code, pick your model up front and stick with it. Try not to walk away in the middle of a big task.
9:10And if you're on an API key and you take long breaks, look at the 1-hour cache setting. Oh, and if your prompt is tiny, it won't cache at all. On Sonnet and Opus 5.5, the minimum is 512 tokens, and anything shorter just skips the cache with no error. So now I'm curious, what's your agent's cash hit rate right now? Did you even know you could check it? Let me know what you see when you check. And if you learned something new, drop a like so this gets in front of more people running agents. Have a blessed day.