Transformer

A deep learning architecture introduced in 2017 that relies entirely on an attention mechanism to draw global dependencies between input and output.

How it works

A Transformer consists of stacked encoder and/or decoder blocks. Each block contains a multi-head self-attention sublayer (which lets every token attend to every other token) and a feed-forward network sublayer, with residual connections and layer normalisation around each. The attention mechanism computes Query, Key, and Value projections for each token; dot products between Queries and Keys determine how much attention each token pays to each other, and these weights are used to aggregate Value vectors.

Why it matters

The Transformer is the foundational architecture of modern AI. Its introduction in 'Attention Is All You Need' (2017) replaced RNNs for NLP, and has since expanded to vision (ViT), audio (Whisper), protein structure (AlphaFold), code (Copilot), and multi-modal models. Every major LLM — GPT, Claude, Gemini, LLaMA, DeepSeek — is a Transformer variant. Understanding the Transformer architecture is the most important prerequisite for understanding modern AI.