Best-of-N Sampling

An inference technique that generates N full candidate responses, scores them with a reward or preference model, and returns only the highest-scoring one.

How it works

At inference time, the model generates N separate responses independently — each with its own sampled tokens — then a reward model or preference classifier scores each response. The response with the highest score is returned to the user and the rest are discarded. The technique trades compute for quality: generating 8 or 16 candidates and selecting the best one is significantly more expensive than a single generation but reliably produces higher-quality outputs.

Why it matters

Best-of-N is one of the simplest ways to use test-time compute to improve model quality without any additional training. It demonstrates that 'thinking longer' at inference time — by generating multiple candidates — yields meaningful gains. This insight has influenced the design of reasoning models like o1 and DeepSeek-R1, which use more sophisticated search and self-evaluation during generation rather than simple re-sampling.