GuidesUnderstanding AI

AI tokens: the unit you pay for, explained

By the Banzai Labs teamUpdated on 3 min read

SummarizethiscontractYour request, in tokensInputOutput€€€€The bill counts both, output costs more
A request split into tokens: the bill counts those going in and those coming out.

What a token is

A language model reads neither letters nor whole words. It splits text into frequent pieces, tokens, then predicts the next token, one at a time.

What the bill counts
  • Input tokensInstructions, history, attached documents, tool descriptions.
  • +
  • Output tokensThe answer written by the model, pricier per token.
  • =
  • The cost of one callMultiplied by the number of calls in the month.

Where tokens go

Item What weighs Lever
System prompt Repeated on every call Keep it short, cache it
History Grows with every message Summarise old exchanges
Documents Whole pages sent Send only the useful passages (RAG)
Answer The most expensive per token Ask for a short format

Three habits to cut the bill

  1. 1MeasureLog input and output tokens for every call.
  2. 2Trim the inputLess history, excerpts rather than whole documents.
  3. 3Pick the modelA small model to sort or summarise, a large one to reason.

To price a full project, see How much an AI project costs. To understand how much a model can read, see the context window.

Frequently asked questions

How many tokens is a word?
A common word is often one token; a long or rare word is several. Languages other than English usually take a few more tokens for the same text.
Why does a conversation get more and more expensive?
On every message, the application sends the whole history back to the model. The tenth message also pays for reading the first nine.
Do images count?
Yes. An image or a PDF page is converted into input tokens, often several hundred per image depending on its size.

Read next

All guides