PEFT (Parameter-Efficient Fine-Tuning)

A family of techniques (including LoRA) designed to fine-tune large pre-trained models by only updating a very small number of extra parameters, saving vast amounts of compute and memory.

How it works

PEFT methods attach small trainable modules to a frozen pre-trained model and update only those. LoRA inserts low-rank matrices into attention layers. Prefix tuning prepends learnable soft tokens to the input. Adapter layers insert small bottleneck modules between Transformer sublayers. Prompt tuning optimises a set of continuous prompt embeddings. All these methods achieve similar effects to full fine-tuning while updating 0.01% to 1% of the total parameters, dramatically reducing GPU memory and training time requirements.

Why it matters

PEFT democratised fine-tuning of large models. Before PEFT, adapting a 7B model required the full GPU resources used to train it. With LoRA, the most popular PEFT method, a 7B model can be fine-tuned on a single consumer GPU. This enabled a flourishing ecosystem of specialised open-source models, made continuous model improvement affordable for smaller organisations, and is the primary mechanism through which enterprises adapt foundation models to their proprietary data without paying for full retraining.