Overview of Linear Digressions: Distillation, or, How to Steal a Model
This episode explains model distillation in large language models: how a larger, more capable “teacher” model can be used to train a smaller, cheaper, faster “student” model. The hosts also dig into the darker side of the technique—how distillation can be used to replicate or “steal” a proprietary model by repeatedly querying it and using its outputs as training data. Along the way, they connect modern LLM distillation to an earlier 2015 paper by Geoffrey Hinton and Jeff Dean, and discuss why preserving a model’s output distribution matters, not just its single best answer.
What Model Distillation Is
Core idea
- A large model generates outputs.
- Those outputs are then used as training data for a smaller model.
- The smaller model learns to mimic the larger one as closely as possible.
Why it’s useful
- Lower cost: smaller models are cheaper to run.
- Lower latency: they can be faster.
- Specialization: a distilled model can be focused on a narrower task instead of carrying the full breadth of a general model.
Examples mentioned
- A coding model may not need deep knowledge of medical treatment.
- A bird-identification model does not need to know how to build a Next.js app.
- Distillation can help create task-specific models that are more efficient.
Distillation as a Way to “Steal” a Model
How the alleged theft works
- A lab with access to a frontier model API sends massive numbers of prompts.
- It collects the responses and uses them to train a competing model.
- At scale, this can approximate the behavior of the original model without training one from scratch.
Why this is hard to detect
- Bad actors can spread requests across:
- many accounts
- many IP addresses
- VPNs or other routing tricks
- This makes it difficult to distinguish normal usage from large-scale extraction.
Forensic clues are imperfect
- Some models may accidentally reveal their origin, e.g. responding “I’m Claude.”
- But this is not conclusive, since similar phrases can appear in public LLM-generated text.
- Simple fingerprinting tricks, like frequent em-dash use, are increasingly unreliable.
The Intellectual Property Debate
- The hosts note that some people argue distillation is fair use or at least not straightforward theft.
- They also point out the broader irony: much of modern AI training is itself built on scraped internet content that was not explicitly licensed.
- So the episode frames “model theft” as part of a larger, messy debate about ownership, data rights, and AI training ethics.
The Distribution Matters, Not Just the Answer
A key conceptual point
- Distillation is not just about copying the most likely answer.
- It can also mean capturing the full probability distribution of what a model might say.
Why that matters
- A model may say “cat” 90% of the time, but sometimes “fox” or “dog.”
- Those less-likely outputs still carry useful information about how the model groups concepts.
- Preserving the distribution helps retain:
- nuance
- generalization
- creativity
- uncertainty
Connection to Hinton and Dean’s 2015 work
- The hosts highlight an early distillation paper by Geoffrey Hinton, Jeff Dean, and another author.
- That work predated transformers and modern LLMs.
- It emphasized using richer internal model information, not just top-line outputs, to transfer knowledge.
Main Takeaways
- Distillation is a practical way to make models smaller, cheaper, and more specialized.
- It can also be used to reverse-engineer or imitate a proprietary model through repeated querying.
- Detecting distillation-based imitation is difficult because model outputs and training data can be noisy and overlapping.
- The most valuable knowledge in a model is often in its probabilistic behavior, not just its single best answer.
- Distillation is relevant not only for deployment, but also for the broader debates around model copying and AI intellectual property.
What’s Coming Next
- The hosts mention that this topic leads into the next episode on reasoning models, where distillation also plays a role.
Notable Ideas and Quotes
- The “teacher-student” metaphor is the central mental model for understanding distillation.
- Distillation is described as both:
- a way to make models more efficient
- and a way to “steal” model behavior by treating the API like a training oracle
Useful Links / References Mentioned
- A 2015 paper by Geoffrey Hinton and Jeff Dean on distillation in neural networks
- The Linear Digressions Substack newsletter, which includes links and extra notes from the episode