Prompt caching, explained — stop paying full price to re-read the same context

If you run an agent loop or send the same long system prompt on every call, you are paying full input price to make the model re-read text it already saw a second ago. Prompt caching fixes that: the provider stores a prefix of your prompt and, on the next call, charges a fraction to read it back instead of the full input rate.

The mechanics are simple, but the billing has a twist most write-ups skip — and the twist is the whole reason to care about how you structure the prompt.

TL;DR

• Cache read (hit): as low as 0.1x the input rate — a ~90% discount on the cached portion.
• Cache write: a one-time premium (~1.25x the input rate on Anthropic's 5-minute window).
• So caching only pays when the same prefix is reused across calls — the first hit already earns back the write. Put static content first, dynamic content last, keep the prefix byte-stable.
• On byesu, caching passes through unchanged — you keep the 0.1x reads on the official upstream.

The billing twist: writes cost more, reads cost far less

A cache entry has two prices, not one:

Event	What happens	Anthropic official structure
Cache write	The prefix is stored (first call, or after it expires)	~1.25x the base input rate (5-min TTL); ~2x for the 1-hour TTL
Cache read (hit)	A later call reuses the stored prefix	0.1x the base input rate
Uncached input	Anything past the cached prefix	1x (normal input)

The write premium is why "just cache everything" is wrong advice. If you write a cache entry and then never read it again, you paid more than if you'd never cached at all. The math only works when the prefix is read enough times to amortize that one write:

The first reuse already pays it back. The write costs ~0.25x more than a normal read (1.25x vs 1x); a single cache hit saves 0.9x (0.1x vs 1x), which more than covers that premium. So from the second call onward you are ahead, and every hit after that is ~90% off the cached tokens. For an agent that loops twenty times over the same system prompt and tool definitions, this is a large, structural cost cut — not a rounding error.

byesu bills the cache-read portion as low as 0.1x, matching this official structure — the gateway does not add a caching markup, so the economics are the same as going direct.

Why agent loops are the ideal case

Prompt caching rewards repetition of an identical prefix. Three workloads hit that pattern hard:

1. Agentic coding / tool loops — Claude Code, Codex, custom ReAct agents. Every step resends the same system prompt, the same tool schemas, and a growing conversation. The static head is identical each turn → cached.
2. Long-document Q&A — you paste a 50-page contract or a large codebase context once, then ask ten questions against it. The document is the prefix; only the question changes.
3. High-volume classification / extraction — a big instruction block + few-shot examples that never change, with only the input row varying.

In all three, the expensive part (the long, stable head) is written once and read many times. That is exactly the shape caching is designed for.

How to actually turn it on

Claude models — explicit cache_control breakpoints

Anthropic caching is opt-in per content block. You mark the end of the stable prefix with a breakpoint; everything up to and including it becomes the cache key.

import anthropic

client = anthropic.Anthropic(
    base_url="https://byesu.com",      # gateway; caching passes through
    api_key="sk-your-own-key",
)

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LONG_STABLE_INSTRUCTIONS,   # your big, unchanging system prompt
            "cache_control": {"type": "ephemeral"},   # <-- breakpoint: cache everything up to here
        }
    ],
    messages=[{"role": "user", "content": todays_question}],   # the only part that varies
)

print(resp.usage)   # cache_creation_input_tokens / cache_read_input_tokens

Rules that make or break the hit:

• Order matters. The cached prefix must come before anything that varies. Static system prompt and tool definitions first; the changing user turn last. If a timestamp or a per-request ID sits inside the prefix, the key changes every call and you write-then-never-read — the worst case.
• Byte-identical. The prefix must match exactly, character for character. A reordered JSON key, a trimmed space, a different date string — any of it busts the cache.
• Minimum size. The cacheable prefix must be roughly 1024+ tokens (a bit more for the smallest models). Below that, there is nothing to cache.
• TTL slides. The default window is ~5 minutes and refreshes on every hit, so a steady stream of calls keeps the entry warm indefinitely; a gap longer than the TTL means the next call pays the write again.

OpenAI models (GPT-5.6 / GPT-5.5) — automatic

OpenAI caches automatically for prompts over ~1024 tokens — no breakpoints. It matches the longest common prefix of recent requests and discounts the cached tokens. Two differences from Claude worth knowing: there is no write premium (creating the cache is free, so there is nothing to waste), and the read discount is smaller than Claude's 0.1x — you still save, just not 90%. Your only job is the same discipline: keep the static content at the front, keep it stable, and let the varying content trail at the end. You will see the effect in prompt_tokens_details.cached_tokens.

Verify you are actually hitting it

Do not assume — read the usage object:

Provider	Write field	Read (hit) field
Claude	cache_creation_input_tokens	cache_read_input_tokens
OpenAI	(implicit)	prompt_tokens_details.cached_tokens

If cache_read_input_tokens (or cached_tokens) is 0 on the second identical call, something in your prefix is not stable — hunt down the moving part before you trust the savings.

On a gateway, caching can silently break — here it doesn't

This is the one gateway-specific thing worth knowing. Some aggregators rewrite requests — reordering fields, stripping cache_control, injecting a per-request header into the prompt body — and that quietly destroys the prefix match. You still get an answer; you just never hit cache and pay full input every time, with no error to tell you.

byesu forwards cache_control and the cached-token usage fields to the official upstream unchanged, so the cache key you build is the cache key that gets used. You keep the 0.1x reads you would get calling the provider directly — on the same official model, with no downgrade. See Pricing & Billing for how the multipliers work.

FAQ

How much does a cache hit save?

The cached portion is billed at as low as 0.1x the base input rate — about a 90% discount on those tokens. Writing the cache costs a one-time premium (~1.25x for the 5-minute window), which the very first cache hit already earns back — so the savings turn positive from the second call onward.

Do I have to change my code?

For Claude, add a cache_control breakpoint to the blocks you want cached and keep everything before it byte-identical. For OpenAI models it is automatic above ~1024 tokens — just keep static content at the front.

Does caching still work through byesu?

Yes. byesu passes cache_control and the cached-token usage fields through to the official upstream unchanged, so you get the same cache economics — reads as low as 0.1x, matching the official structure.

How do I confirm it is working?

Check cache_read_input_tokens (Claude) or prompt_tokens_details.cached_tokens (OpenAI) on the response. Zero on a repeated call means your prefix is not stable or the cached content moved.

---

Cache multipliers describe the official upstream pricing structure that byesu passes through; the console shows your actual billed price per request. Official provider rates cited for context belong to their respective providers.