Temperature

A parameter in language models that controls the randomness or creativity of the output. Higher values lead to more random outputs.

How it works

Before applying softmax, each logit is divided by the temperature value T. At T=1 (default), the raw model probabilities are used. At T<1 (e.g., 0.2), logits are amplified, making the distribution sharper and the model more likely to pick the highest-probability token (more deterministic). At T>1 (e.g., 1.5), logits are suppressed, flattening the distribution and making low-probability tokens more likely to be selected (more random). T=0 is equivalent to greedy decoding.

Why it matters

Temperature is the primary control for the creativity-vs-accuracy trade-off in LLM generation. Low temperature is better for factual Q&A, code generation, and structured tasks where the correct answer is clear. High temperature produces more varied, surprising outputs suitable for brainstorming, creative writing, and data augmentation. Almost every LLM API exposes temperature as a key parameter, and tuning it is often the fastest way to improve output quality for a specific application.