When a company distills knowledge from a competitor's AI, it's not just scraping pre-training data. It's a highly efficient process of extracting the model's intelligence, reasoning patterns, and skills. This is more akin to an apprentice directly interacting with and learning from a world-class expert than simply reading the same textbooks the expert used.

Large, centralized AI models are vulnerable to 'distillation attacks,' where a smaller model can be trained cheaply by querying the larger one. This technical reality, combined with the moral hypocrisy of creators restricting copying after scraping the internet, strongly suggests a future dominated by decentralized, open-source models.

While techniques like model distillation can reduce costs for near-frontier AI capabilities, this hasn't dampened demand for the absolute best models. The market shows very little desire for the third-best model, but exceptional demand for the top-performing one for any given task, demonstrating a winner-take-all dynamic.

China is gaining an efficiency edge in AI by using "distillation"—training smaller, cheaper models from larger ones. This "train the trainer" approach is much faster and challenges the capital-intensive US strategy, highlighting how inefficient and "bloated" current Western foundational models are.

The common practice of model distillation suggests that AI capabilities will eventually be commoditized. As smaller models can cheaply mimic larger ones, differentiation will shift away from raw performance to product integration and price, likely triggering a massive price war among providers.

In his trial against OpenAI, Elon Musk admitted under oath that using one AI model to train another—a practice known as distillation—is something 'all the companies do.' This confirms that a legally and ethically gray practice is widespread across the industry.

The process of 'distillation' involves using a large, expensive LLM to perform a task repeatedly. The resulting prompts and responses then become the training data to create a smaller, specialized, and much cheaper Small Language Model (SLM) that can perform that specific task, potentially saving 90% on inference costs.

The public-facing models from major labs are likely efficient Mixture-of-Experts (MOE) versions distilled from much larger, private, and computationally expensive dense models. This means the model users interact with is a smaller, optimized copy, not the original frontier model.

Unable to build frontier models from scratch, some Chinese companies gain a competitive edge by using "scale distillation." This involves training smaller, open models on the outputs of larger, proprietary US models, effectively piggybacking on American R&D to create capable, low-cost alternatives.