GuidesUnderstanding AI

Prompt caching: paying less for what does not change

By the Banzai Labs teamUpdated on 2 min read

Written to cacheRead from cacheRead from cacheSame system promptQuestionThe fixed part is re-read at a reduced price
The system prompt, identical on every call, is written to cache once and then re-read for less.

The principle

A fixed start, a changing end
Without cache
  • Instructions re-read at full price on every call
  • The reading time repeats
With cache
  • Written once, re-read at a reduced price
  • An answer that starts sooner
  • Only the question is read at full price

How to enable it with Claude

A cache_control marker on the last fixed block (system prompt, documents) shows how far to cache:

system: [
  { type: 'text', text: instructionsAndDocuments, cache_control: { type: 'ephemeral' } },
],

The response’s usage field shows the tokens written to and read from cache: that is where you check it works.

The rules

  • Fixed first, variable after. Instructions and documents at the top, the question at the end.
  • Nothing changing at the start. No date or ID in the system prompt.
  • A minimum size. A very short prompt is not cached; the threshold depends on the model.

To put numbers on it: conversation cost. For the other levers: capping your AI API bill.

Frequently asked questions

How much does it save?
The part read from cache is billed at a fraction of the normal price (around a tenth at Anthropic, less on recent models). The first write costs a little more. Exact prices are on the provider's pricing page.
How long does the cache last?
At Anthropic, five minutes by default, renewed on every read; a one-hour duration is available as an option. Steady traffic therefore keeps the cache warm.
Why is my cache never used?
Usually something changes at the start of the prompt: the date, an ID, the order of the tools. Caching only works if the start is byte-for-byte identical. A prompt that is too short is not cached either.

Read next

All guides