What is AI model distillation? [Explainer]

The cheapest way to build a capable AI model is to let somebody else’s expensive one do the teaching. That’s the whole fight, compressed into one sentence.

It broke open in January of last year, when Chinese AI startup DeepSeek released R1, its debut model. R1 approached frontier-model capability on some key benchmarks. More to the point, it was rumored to have been built at a fraction of what the biggest U.S. labs had spent on their flagships.

Not long after the release, OpenAI accused DeepSeek of distilling the model behind ChatGPT to train DeepSeek. A little over one year later, Anthropic accused DeepSeek and fellow Chinese labs Moonshot and MiniMax of doing the same thing with Claude. In June, Anthropic accused Chinese tech giant Alibaba of “illicitly” using Claude for distillation purposes.

Read that word again. Illicitly, not illegally.

The legal rules don’t exist yet, but the politics do

The legal rules around distillation are ill-defined, which is true of most things in the current AI industry. The Trump administration has still vowed to crack down on Chinese labs distilling U.S.-made models, framing the AI race between the two countries as a matter of supreme geopolitical import.

“AI distillation, when legitimately used to produce smaller, lighter-weight models from more advanced systems, is a vital part of [the AI] ecosystem,” Michael Kratsios, White House Office of Science and Technology Policy director, wrote in an April memo. “Industrial distillation activities that aim to systematically undermine American research and development and access proprietary information, however, are unacceptable.”

So the White House isn’t calling the technique itself dirty. It’s drawing a line around who does it, at what scale, and to whom.

What the word actually borrows from

Distillation, in the older sense, is an alchemical process that pulls a small valuable kernel out of a lot of otherwise useless material. Boil a fermented liquid, capture the ethanol vapor, cool it. The pure alcohol gets separated, extracted and sold.

AI model distillation, also known as knowledge distillation, does something structurally similar. It extracts valuable training data from a larger, pretrained model instead of grinding out that knowledge from scratch.

And the grind is the expensive part. Ordinarily a model has to run a long trial-and-error process to build up facts that a human child treats as self-evident: how to tell a cat from a dog, that 2+2 will always equal four, that the word “tree” refers to one of those tall leafy things outside. That learning phase, as it’s most commonly carried out, is extremely computationally expensive.

Companies have been willing to spend the GDP of a small country doing it that way, because it’s widely believed to be the surest path to artificial general intelligence. Whether that belief holds up is still very much a multibillion-dollar open question.

Teacher, student, and the part people get wrong

Distillation pairs a smaller “student” model with a flagship “teacher” model. Across many interactions, the student gradually learns to predict what the teacher’s responses will be, modeling itself after the teacher. The student is still learning. It’s just doing a much cheaper, more streamlined version of what the teacher had to do.

Here’s the detail that gets flattened in most coverage. A good teacher model doesn’t answer in yes-or-no terms. It shows its reasoning.

Say a teacher model learned to identify motorcycles by examining millions of images scraped from social media and other online databases, helped along by humans filling out millions of CAPTCHAs that teach it to separate motorcycles from similar-looking objects. The student doesn’t just watch the teacher stamp “Yes” or “No” on pictures. It watches the teacher deduce.

The teacher might conclude the object in an image has an 85% chance of being a motorcycle, a 12% chance of being an e-bike, a 2.7% chance of being a donkey and a 0.3% chance of being Mr. Snuffleupagus. Those are called soft labels, and they’re what builds the student’s own reasoning and statistical prediction skills.

Two analogies that hold up

Think of a flagship model as a native New Yorker whose knowledge of the city came from countless wanderings and wrong turns. A distilled model is the friend visiting for the first time, shadowing her around town and sketching a crude map along the way.

Or take the Wright Brothers, who flew their first flying machine in 1903 after many failed attempts and a long history of other innovators who tried the same thing and didn’t manage it. Once they succeeded, everyone else could build off their model. The hard learning was done.

Which is why most experts will tell you there’s nothing necessarily immoral about model distillation. Building on the innovations of others is the foundation of scientific progress. “If I have seen further,” as Isaac Newton put it, “it is by standing on the shoulders of giants.” You can even make the case that distillation is a net good, since it spreads the ability to build powerful AI to people who could never fund a flagship.

The safety layer doesn’t always come along

Geopolitics isn’t the only worry here, and it may not be the most serious one.

AI is an inexact science. Train a flagship model on close to the entire corpus of the internet and it picks up benign pattern-recognition rules, like sentence structure or how to spot a cat. It also picks up deeply ingrained human biases. With enough compute, you can slowly train models to spot those biases and steer clear of them. Models can also be fine-tuned to notice when a user wants something dangerous, a recipe for a bioweapon being the standard example, and refuse.

Those safeguards don’t always survive distillation. A student model tends to pick up foundational behaviors while missing the deeper layer of safety knowledge the teacher relies on to decide when not to indulge a request. Alignment gets lost in translation.

Bad habits compound

The same leakiness applies to bias. Even conscientious parents pass on habits they never meant to teach, and a student model can pick up skewed perspectives about the world from its teacher without anyone noticing.

Then that student gets used to train another model. The original bias goes down another generation, and another, further from the flagship that produced it and further from anyone who could audit where it came from.

When you’re evaluating a small, cheap model that punches above its size, the useful question isn’t how it performs on benchmarks. It’s whose knowledge it inherited, and what it quietly failed to inherit along with it.