RLHF (Reinforcement Learning from Human Feedback)

A technique used to train AI models by incorporating human feedback to reward desired behaviors.

How it works

RLHF has three stages. First, supervised fine-tuning trains the model on high-quality demonstration data so it can produce reasonable responses. Second, human annotators rank pairs of model outputs by preference, and these rankings are used to train a reward model that predicts human preference scores. Third, the language model is fine-tuned using PPO (or a similar RL algorithm) to maximise the reward model's scores, nudging it to produce outputs that humans prefer.

Why it matters

RLHF is what made ChatGPT feel helpful rather than just statistically plausible. Raw language models trained only on next-token prediction will happily produce harmful, dishonest, or unhelpful content if that's what's in the training data. RLHF teaches models to follow instructions, refuse inappropriate requests, and prioritise user intent. It is the core alignment technique used by OpenAI, Anthropic, Google, and Meta — making it arguably the most commercially impactful ML training technique of the current era.