Cache a long system prompt and documents

Approach

Put stable content first (tools, system prompt, reference documents) and mark the end of the stable part with cache_control. Everything that changes per request (the question, user data, timestamps) goes after it. Check usage.cache_read_input_tokens to confirm hits.

Solution

import anthropic

client = anthropic.Anthropic()
POLICY_HANDBOOK = open("handbook.md", encoding="utf-8").read()     # ~40k tokens, changes monthly


def ask(question: str):
    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=800,
        system=[
            {"type": "text", "text": "You answer HR questions using only the handbook below."},
            {"type": "text", "text": f"<handbook>\n{POLICY_HANDBOOK}\n</handbook>",
             "cache_control": {"type": "ephemeral"}},          # cache everything up to here
        ],
        messages=[{"role": "user", "content": question}],         # varies, so after the cached prefix
    )
    u = response.usage
    print(f"cache write {u.cache_creation_input_tokens}, cache read {u.cache_read_input_tokens}, "
          f"uncached input {u.input_tokens}")
    return response.content[0].text

Pitfalls

• Putting a timestamp or user name inside the cached part: the prefix changes and nothing is ever reused.
• Expecting hits on the first request; the first call writes the cache.
• Prefixes shorter than the model's minimum cacheable length are not cached.

Variations

• Use "ttl": "1h" in cache_control for traffic with longer gaps between requests.
• Pass top-level cache_control={"type": "ephemeral"} to cache automatically up to the last cacheable block.

Further reading: Prompt caching