The term „distillation attacks“ is a terrible name for what’s currently taking place. Yes, certain Chinese labs are hacking or jailbreaking APIs in an effort to draw more intelligence from model services — preventing this is crucial to preserving America’s advantage in AI capabilities. Calling this a “distillation attack” will permanently link the entire concept of distillation to this malicious behavior, even though distillation itself is a fundamental technique required to spread AI capabilities widely across academia and the economy. We already went through a similar shift in terminology during the open-source versus open-weights debate. All the terms have simply collapsed into “open models” – very few people in the broader AI community actually understand the difference between open-source and open-weights. Terminology is important, because less informed individuals who remain concerned about — and help shape — technology are constrained by the different terms they employ. If we aren’t cautious about how we discuss distillation, many could start viewing this versatile R&D technique for developing new models as something bordering on corporate manipulation or even criminal activity. Share. I’ve recently published a more technical analysis examining the real impact of state-of-the-art distillation techniques on leading Chinese models. This article follows up by advocating for caution against any rushed policy measures aimed at restricting those methods. Anthropic recently published a blog post describing „distillation attacks“ carried out by three Chinese labs. The labs employed a technique known as „distillation,“ in which a weaker model is trained on the outputs of a more powerful one.
Interconnects AI