EOS (End of Sequence)

A special token that signals the language model to stop generating further tokens for the current response.

How it works

During training, EOS tokens are appended to the end of every training example. The model learns that after generating an EOS token, it should stop. At inference time, the decoding loop checks after each generated token whether the selected token is EOS. If it is, generation halts and the response is returned. Some models and APIs also support setting custom stop sequences — arbitrary strings that trigger early termination — in addition to the standard EOS token.

Why it matters

Without a reliable stop signal, a language model would continue generating tokens indefinitely (or until a hard token limit is hit), often producing incoherent rambling after the natural end of a response. EOS is what makes conversations feel natural and responses self-contained. It also enables structured output: models fine-tuned to generate JSON or XML can learn to emit EOS immediately after the closing bracket, making output parsing reliable.