LLaMA

A family of large language models released by Meta.

How it works

LLaMA models use the standard decoder-only Transformer architecture but with several efficiency improvements adopted from research: RoPE positional embeddings, SwiGLU activation functions, and RMSNorm instead of LayerNorm. They are pre-trained on large, carefully curated public datasets rather than proprietary web crawls. Meta releases model weights openly under research or commercial licences, enabling the community to fine-tune and run them locally without API access.

Why it matters

LLaMA ignited the open-source LLM ecosystem. When LLaMA 1 leaked in early 2023, it triggered an explosion of fine-tuned variants (Alpaca, Vicuna, WizardLM) that demonstrated high-quality conversational AI could run on consumer hardware. LLaMA 2 and 3 formalised Meta's commitment to open weights, establishing a credible open-source alternative to closed models from OpenAI and Anthropic and enabling innovation at organisations that cannot afford API costs at scale.