DeepSeek

A Chinese AI lab and model family known for open technical reports on MoE transformers and Multi-Head Latent Attention (MLA), including DeepSeek-V2, V3, and related chat / reasoning assistants.

How it works

DeepSeek publishes unusually detailed technical reports alongside their model releases. Their core architectural innovations include Multi-Head Latent Attention (MLA) — which compresses the KV cache into a latent vector to reduce memory — and a fine-grained Mixture of Experts design that routes each token to a small subset of experts rather than a single large FFN. Their training pipeline also uses auxiliary-loss-free load balancing to keep expert utilisation even without hurting model quality.

Why it matters

DeepSeek's releases demonstrated that frontier-quality models can be built outside the US at a fraction of the reported compute cost of models like GPT-4, challenging assumptions about who can compete at the AI frontier. Their open technical reports have been invaluable to the research community, sharing implementation details that most labs keep proprietary and directly influencing designs at other organisations.