Simulated Annealing

A probabilistic technique for approximating the global optimum of a given function, inspired by metallurgy.

How it works

Simulated annealing mimics the physical process of heating a metal and slowly cooling it so atoms settle into a low-energy crystalline structure. In optimisation, the algorithm starts by accepting both better and worse solutions with some probability. As a 'temperature' parameter decreases over time, the probability of accepting worse solutions drops, causing the search to settle toward a good optimum. This prevents it from getting stuck in local minima unlike plain hill-climbing.

Why it matters

Many real-world optimisation problems — chip layout, scheduling, hyperparameter search — have enormous search spaces with countless local optima. Simulated annealing's controlled randomness allows it to escape those traps and find near-global solutions without needing gradient information, making it valuable wherever gradient-based methods cannot be applied.