GPT (Generative Pre-trained Transformer)

A series of large language models developed by OpenAI.

How it works

GPT models are decoder-only Transformers pre-trained on large text corpora using a next-token prediction objective — given all previous tokens, predict the next one. This self-supervised training requires no human labels and scales efficiently with more data and compute. After pre-training, the models are fine-tuned with RLHF to follow instructions and be helpful. Each GPT generation (GPT-2, GPT-3, GPT-4) scaled up model size, data, and compute, yielding qualitative capability leaps.

Why it matters

The GPT series pioneered the paradigm of large-scale language model pre-training followed by fine-tuning, which is now standard across the entire AI industry. GPT-3 was the first model to demonstrate that scale alone could produce surprisingly general-purpose capabilities. GPT-4 raised the bar for reasoning and multi-step problem solving. The architectural and training choices OpenAI made in the GPT series have been replicated and extended by virtually every major AI lab.