Softmax

A function that turns a vector of raw scores (logits) into a probability distribution that sums to one, used when sampling the next token.

How it works

The softmax function takes a vector of raw logits (one per vocabulary token, often 50,000+ values) and exponentiates each value, then divides by the sum of all exponentiated values. This maps any real-valued vector to a valid probability distribution where all values are positive and sum to one. The output probabilities are then used to sample the next token, either greedily (pick the max), via top-k/top-p sampling, or according to the temperature-scaled distribution.

Why it matters

Softmax is the final gate between a model's internal computation and the token it produces. The temperature parameter works by dividing all logits by a temperature value before softmax, which sharpens (low temperature) or flattens (high temperature) the distribution and thus controls output diversity. Understanding softmax is key to understanding how sampling strategies, temperature, and logit biasing techniques work to shape LLM outputs.