Hallucination

When an AI model generates false, fabricated, or nonsensical information but presents it as a fact.

How it works

LLMs generate text by predicting the next most probable token given the previous context. They do not have a separate fact-checking mechanism — if a plausible-sounding but false continuation has high probability, the model will produce it. Hallucinations are most likely when the model lacks training data about a topic, when questions are ambiguous, or when the model is pressured to produce a specific form of answer (e.g., a citation) even when it doesn't have the information to do so accurately.

Why it matters

Hallucination is the most significant barrier to deploying LLMs in high-stakes applications like medicine, law, and finance. A model that confidently generates false drug interactions, incorrect legal precedents, or fabricated financial figures can cause serious harm. Addressing hallucinations drives significant research into RAG (grounding models on retrieved documents), fine-tuning for calibration, and output verification systems that check LLM claims against authoritative sources.