Few-shot Learning

The practice of feeding a machine learning model with a very small amount of training data, contrary to the normal practice of using a large amount.

How it works

In the context of LLMs, few-shot learning works by including a small number of input-output examples directly in the prompt before the actual query. The model generalises from these examples without any weight updates. This is also called in-context learning. For traditional ML, few-shot learning refers to training paradigms like Prototypical Networks or MAML that are explicitly designed to generalise from very few labelled examples by learning a good feature space during pre-training.

Why it matters

Labelled training data is often expensive, slow, or impossible to collect at scale — especially in specialised domains like medicine, law, or low-resource languages. Few-shot learning dramatically reduces this dependency. For LLMs, it means a model can be adapted to new tasks at inference time purely through prompt design, without any fine-tuning. This versatility is one of the key reasons LLMs are so broadly applicable across industries.