What is prompt caching?

Every time you call an API, the model re-reads your entire prompt — the system instructions, the examples, the documents, all of it — and you are billed for every token, every time. Even though most of it was identical last time.

Prompt caching lets the vendor keep the processed form of that repeated prefix around, so subsequent requests skip most of the work and most of the cost.

The economics

The hit is the famous part: about 90% off. The write is where vendors quietly differ, and it is left out of every summary. Comparing like for like on a $5.00-per-million input rate, as published on 17 July 2026:

Two different deals are hiding in that table.

Where a write premium exists — Anthropic, and OpenAI's newest family — caching genuinely costs more than not caching, up front. You are buying an option, and it only pays if you exercise it. A cache that expires before anyone reads it is a pure loss. The break-even is early, from the second or third read depending on duration, but it is not zero; we work it through in how to cut your LLM API bill.

Where it does not — OpenAI's gpt-5.5 and gpt-5.4 families publish no cache write charge at all — caching is a free discount with no downside. Turn it on and stop thinking about it.

So the first question is not "should I cache?" but "does my model charge me to?" Nobody puts that on the feature page.

What can be cached

The mechanic that matters: caching works on a prefix. The cached portion has to be the beginning of your prompt, and it has to be byte-identical to last time.

That has a design consequence people hit late. Put the stable material first — system prompt, instructions, examples, reference documents — and the variable material last. A timestamp, a session id, or a user name near the top of your prompt invalidates everything after it, and you are paying full price while believing you are caching.

Order your prompt: static → semi-static → dynamic.

What it is not

• It is not memory. Caching makes re-reading cheaper. It does not make the model remember your last conversation.
• It is not a quality change. The output is the same. This is a billing and latency optimisation, nothing more.
• It is not permanent. Caches expire — commonly five minutes or an hour, priced differently. Pick the duration from your real traffic pattern.
• It is not automatic everywhere. Some vendors cache implicitly, others need explicit markers. Check the docs for the one you use.

Why you should care

Because for a typical application, the repeated prefix is most of the bill. A long system prompt plus a few reference documents, sent on every request, dwarfs the user's actual question.

Caching is usually the single largest saving available that does not require changing model or accepting worse output — which makes it the rare optimisation with no quality trade-off attached. The only cost is thinking about the order of your prompt.