The idea
Many calls share the same start: a long system prompt, tool definitions, a reference document. Prompt caching lets the provider keep the processed state (the KV cache) for that prefix for a few minutes. Later calls that begin with exactly the same tokens read from the cache instead of recomputing.
Cached input tokens are billed at a fraction of the normal price, and time to first token drops, sometimes sharply for long prefixes. Writing to the cache may cost slightly more than a normal input token the first time.
Caching only matches an identical prefix. Put stable content first (system prompt, tools, documents) and variable content last (the user’s question). A timestamp at the top of the prompt breaks every cache hit.
For chat over a long document or agents with many tools, caching is often the single biggest cost saving available.
Words to know
- Cache hitThe prefix was found and reused.
- Cache breakpointWhere you mark the end of the cacheable prefix.
- TTLHow long the cache entry lives.
Try it without code
Spot the cache in usage numbers
- Run the code below twice within a minute.
- Compare cache_creation_input_tokens on the first run with cache_read_input_tokens on the second.
Use it in code
Mark a long document as cacheable and inspect usage.
import anthropic
client = anthropic.Anthropic()
handbook = open("handbook.txt").read() # a few thousand tokens or more
def ask(q):
msg = client.messages.create(
model="claude-sonnet-5-5", max_tokens=300,
system=[{"type": "text", "text": "Answer using the handbook only."},
{"type": "text", "text": handbook, "cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user", "content": q}],
)
u = msg.usage
print(q, "| write:", u.cache_creation_input_tokens, "read:", u.cache_read_input_tokens)
ask("How many leave days do I get?")
ask("What is the expense limit?")Measure cost and latency for 10 questions over one document with and without caching. Record the saving as a percentage.
Common mistakes
- Putting dynamic values (dates, user names) before the cached content.
- Prefixes shorter than the provider’s minimum cacheable length.