What is temperature in AI models?
Temperature sets how much risk a model takes when picking the next word. Low values give consistent answers, high values varied ones.
What is temperature?
At every step a language model computes a list of candidates for the next token and a probability for each. Temperature adjusts how those probabilities are used. At low temperature the model almost always picks the most likely candidate. At high temperature it gives less likely candidates a chance too, so answers vary and sometimes surprise.
Take the sentence "For breakfast most people eat ...". Low temperature picks "eggs" or a similar top choice nearly every time. High temperature sometimes says "pancakes", sometimes "porridge".
Which value for which job?
Most APIs accept values between 0 and 1 or 0 and 2. Jobs with one right answer, such as data extraction, coding and classification, use low values. Jobs that need variety, such as ad copy, name ideas or brainstorming, try higher values. Push the value too high and the text loses coherence.
Does temperature 0 always give the same answer?
Usually, but not guaranteed. Small numerical differences in hardware and software mean the same prompt can occasionally produce different output even at zero. If you need exact repeatability, store the output and reuse it.
Temperature and hallucination
Low temperature makes a model consistent, not correct. If the model believes a wrong fact is highly likely, it will repeat it at temperature zero as well. The way to reduce hallucination is to give the model the right information, for example with RAG.
A related setting is top-p: the model only chooses among the strongest candidates whose combined probability reaches a threshold. Usually only one of the two is adjusted.