We scan new podcasts and send you the top 5 insights daily.
The controversial practice of AI 'distillation' is not IP theft but a modern form of competitive benchmarking. It's akin to how early Google submitted queries to Yahoo to compare and improve its own search results. The focus is on learning from a competitor's public output, not stealing their underlying software or code.
Accusations that Chinese labs cheat by copying US models are misleading. The practice, known as distillation, is common across the industry (including by Elon Musk's xAI) and academia. Now, with Chinese labs dominating open source, American startups are increasingly building on top of Chinese models.
When a company distills knowledge from a competitor's AI, it's not just scraping pre-training data. It's a highly efficient process of extracting the model's intelligence, reasoning patterns, and skills. This is more akin to an apprentice directly interacting with and learning from a world-class expert than simply reading the same textbooks the expert used.
As more of the internet and code repositories are generated by leading AI models, any new model trained on this public data inadvertently "distills" the knowledge and quirks of those proprietary systems. This blurs the line between original training and outright copying.
In his trial against OpenAI, Elon Musk admitted under oath that using one AI model to train another—a practice known as distillation—is something 'all the companies do.' This confirms that a legally and ethically gray practice is widespread across the industry.
Hugging Face's CEO dismisses the controversy around distillation, framing it as a widespread technique that offers only a marginal boost. It doesn't determine a model's fundamental quality—'if you suck, you suck with or without distillation'—and he questions the merit of 'unfair competition' claims from dominant, trillion-dollar companies.
Chinese labs use 'smart distillation,' a sophisticated technique where a frontier model acts as a 'teacher' to guide a smaller model's judgment and data labeling. This is viewed as a legitimate and efficient catch-up method, distinct from simply copy-pasting answers.
Comparing AI distillation to Ford taking apart a Tesla is a flawed analogy. Reverse-engineering a legally purchased product is generally legal under trade law. However, large-scale distillation of an AI model via API calls typically violates the provider's terms of service, creating a distinct legal challenge around digital IP and contract law.
AI companies protest when competitors "distill" their models, calling it a violation. This stance is deeply ironic, as it mirrors the complaints of artists and creators whose work was scraped without permission to build the original models. The industry fails to acknowledge this double standard.
A new battle line in AI is emerging around model distillation. US officials are framing "covert industrial distillation," like Moonshot AI's alleged activities, as unacceptable IP theft. This is distinct from legitimate distillation used to create smaller, efficient open-source models, setting the stage for future regulation and trade disputes.
Unable to build frontier models from scratch, some Chinese companies gain a competitive edge by using "scale distillation." This involves training smaller, open models on the outputs of larger, proprietary US models, effectively piggybacking on American R&D to create capable, low-cost alternatives.