Backpropagation

A method used in artificial neural networks to calculate the error contribution of each neuron after a batch of data is processed.

How it works

After a forward pass computes the network's prediction and the resulting loss, backpropagation applies the chain rule of calculus to compute the gradient of the loss with respect to every weight in the network. Starting from the output layer and working backwards, each layer's gradients are calculated using the gradients of the layer ahead of it. These gradients then tell an optimiser like Adam or SGD how to adjust each weight to reduce the loss.

Why it matters

Backpropagation is the algorithm that makes training deep neural networks practical. Before its widespread adoption in the 1980s and 90s, there was no efficient way to learn in multilayer networks. Every modern AI model — from image classifiers to large language models — owes its ability to learn from data to this single algorithm. Improvements like gradient clipping, mixed precision, and modern optimisers all build on top of it.