SFT (Supervised Fine-Tuning)

Fine-tuning a base model on curated instruction–response pairs so it learns to act like a helpful assistant rather than only continuing raw text.

How it works

SFT takes a pre-trained base language model and continues training it on a high-quality dataset of (instruction, response) pairs using standard next-token prediction loss. The responses are written or selected by humans (or strong AI models) to exemplify ideal assistant behaviour: helpfulness, clarity, and safety. Because the dataset is carefully curated rather than scraped from the web, even a relatively small SFT dataset (thousands to tens of thousands of examples) can substantially change the model's behaviour.

Why it matters

SFT is the first and most important step in turning a raw pre-trained model into a usable assistant. Without SFT, a base model will continue web text rather than respond to instructions. It is the foundation that makes RLHF and DPO possible — those techniques need a model that can already produce coherent instruction-following outputs to refine. Open-source SFT datasets like Alpaca, FLAN, and OpenHermes have been critical enablers of the community's ability to create high-quality instruction-following models.