GuidesUnderstanding AI

Temperature and top‑p: the settings of an AI model

By the Banzai Labs teamUpdated on 2 min read

01Low: the same answerHigh: varied answers
Low temperature: the same answer every time. High: answers that vary.

What temperature controls

For each token, the model has a list of possible continuations with their probability. Temperature sharpens or flattens that list. Low, it almost always takes the favourite. High, it dares the next ones.

The other settings

  • Top-pKeeps only the continuations that add up to a given share of probability.
  • Max lengthCuts the answer beyond a number of tokens. A cost ceiling.
  • Stop sequencesA text that, if it appears, stops generation.

Which value for which use

Use Temperature Why
Extract fields, classify Low The same document must give the same result
Customer service answers Low to medium Consistent, yet natural
Write a text Medium Varied without drifting
Suggest ideas High Diversity is the goal

What temperature does not do

It does not make an answer more accurate. A mistake can be repeated at low temperature. Reliability comes from sources and checks: see hallucinations.

Frequently asked questions

Temperature 0: is the answer always identical?
Almost, with no absolute guarantee. For a strictly reproducible result, validate the output in code rather than rely on temperature.
Should temperature and top-p be tuned together?
No. Providers advise changing one or the other, not both.
Do all models accept these settings?
No. Some recent so-called reasoning models ignore or fix these parameters. Check the documentation of the model you pick.

Read next

All guides