Epoch

One complete pass of the training dataset through the algorithm during the machine learning training process.

How it works

Training data is divided into mini-batches. One epoch is completed when the model has seen every example in the dataset exactly once — i.e., it has processed all mini-batches. After each mini-batch, the optimiser updates the model's weights. After each epoch, the training and validation losses are typically logged. Training continues for multiple epochs until the validation loss stops improving or begins to increase (signalling overfitting), at which point early stopping may be triggered.

Why it matters

Epochs are one of the most important hyperparameters in training. Too few epochs leave the model underfitted — it hasn't learned the data well. Too many epochs cause overfitting. For large language models, a single epoch over a trillion-token dataset is often enough due to the sheer volume of data — revisiting the same data would be wasteful and counterproductive. Understanding epochs is fundamental to reasoning about training dynamics and reproducibility.