Model Collapse

A phenomenon where an AI model trained heavily on AI-generated data begins to degrade in quality and lose its representation of the real-world distribution.

How it works

When a model generates synthetic data that is used to train the next generation of models, errors and biases in the original outputs get amplified. Rarer, more complex patterns in the original human data are underrepresented in the synthetic data (because the generator model assigns them low probability), so successive models trained on more and more synthetic data progressively lose the ability to produce those patterns. This causes quality to drift and variance to collapse — the model becomes overconfident and repetitive.

Why it matters

As AI-generated content floods the internet, future training datasets will inevitably contain an increasing proportion of synthetic data. Without careful curation and deduplication, models trained on these datasets risk model collapse — a feedback loop that degrades quality over successive model generations. It is a major practical concern for labs maintaining continuous training pipelines and for the long-term quality of AI outputs across the industry.