Multi-Query Attention (MQA)

An attention design that shares Key and Value projections across attention heads, shrinking the KV cache and speeding up chat inference compared with full multi-head attention.

How it works

Standard multi-head attention (MHA) uses separate Q, K, and V projection matrices per head. In MQA, all attention heads share a single K projection and a single V projection, but each head retains its own Q projection. This means the KV cache stores one K and one V per token regardless of the number of heads, reducing KV cache size by a factor of num_heads. Grouped Query Attention (GQA) is a compromise: it groups heads to share K and V in groups, balancing MQA's memory savings with MHA's quality.

Why it matters

MQA and GQA are the dominant inference-time optimisations for reducing KV cache memory in production LLMs. LLaMA 2 introduced GQA; Mistral, Gemma, LLaMA 3, and most modern models use GQA as standard. The KV cache reduction from GQA/MQA directly translates to higher batch sizes (serving more users per GPU) or longer context windows at the same memory budget. This is why virtually every modern open-source LLM uses GQA rather than full MHA.