Logits

Raw, unnormalized scores a model assigns to each possible next token before softmax turns them into probabilities.

How it works

At the final layer of a language model, the hidden state for the current position is projected through an unembedding matrix (typically the transpose of the embedding matrix) to produce a vector of size |V| (vocabulary size). Each value in this vector is a logit — a raw score indicating how strongly the model predicts that token as the next one. These logits are then passed through softmax to produce a probability distribution. Logit processors can modify logits before softmax to implement sampling strategies, biases, or constraints.

Why it matters

Logits are the interface between a model's computation and its output. Decoding strategies (temperature, top-p, top-k), repetition penalty, and classifier-free guidance all work by manipulating logits before softmax. In mechanistic interpretability, analysing the logit lens (applying the unembedding matrix to intermediate layer representations) reveals what the model 'predicts' at each layer, providing insight into how information is processed through the network depth.