Glossary
Distillation
Training a smaller model to imitate a larger one, so it is cheaper to run.
Distillation trains a smaller "student" model on the outputs of a larger "teacher" model. The student learns to approximate the teacher's answers at a fraction of the cost of running the larger model.
Developers routinely distill their own models to offer cheaper, faster versions. Distillation also comes up in disputes, when one company is accused of training on another company's model outputs in breach of its terms of service.
A distilled model usually gives up some capability in exchange for speed and cost. A claim that a small model "matches" a larger one should say on which tasks.