When a company distills knowledge from a competitor's AI, it's not just scraping pre-training data. It's a highly efficient process of extracting the model's intelligence, reasoning patterns, and skills. This is more akin to an apprentice directly interacting with and learning from a world-class expert than simply reading the same textbooks the expert used.

Contamination in coding benchmarks is subtle. Instead of just spitting out a known solution, models like GPT-5.2 use implicit knowledge from their training data (e.g., popular codebases) to reason about unstated requirements. This makes it hard to distinguish true capability from memorization, as the model's 'chain of thought' appears logical while relying on leaked information.

Large, centralized AI models are vulnerable to 'distillation attacks,' where a smaller model can be trained cheaply by querying the larger one. This technical reality, combined with the moral hypocrisy of creators restricting copying after scraping the internet, strongly suggests a future dominated by decentralized, open-source models.

In his trial against OpenAI, Elon Musk admitted under oath that using one AI model to train another—a practice known as distillation—is something 'all the companies do.' This confirms that a legally and ethically gray practice is widespread across the industry.

The public-facing models from major labs are likely efficient Mixture-of-Experts (MOE) versions distilled from much larger, private, and computationally expensive dense models. This means the model users interact with is a smaller, optimized copy, not the original frontier model.

As developers increasingly use AI coding assistants like Claude Code, they flood public repositories like GitHub with high-quality, AI-generated outputs. This effectively turns the internet into a massive, unavoidable training dataset for competing models, making it difficult to police "distillation" as a violation of terms.

API providers like Anthropic struggle to differentiate between users distilling models for competitive purposes and those conducting large-scale evaluations. Both activities generate similar high-volume, repetitive API calls, creating a detection challenge that also raises user privacy concerns.

A key reason for restricting access to new AI models is the threat of 'distillation.' Malicious groups can use thousands of consumer accounts to systematically query a model, effectively reverse-engineering its capabilities. This 'professionalized fraud' can then be used to create powerful open-source alternatives, undermining the entire closed-source business model and security strategy.

Microsoft chose not to use distillation from superior models like OpenAI's to train its new MAI-1 model. Mustafa Suleiman argues that while distillation provides short-term gains, it prevents a model from ever surpassing its 'teacher,' hindering the development of a world-class lab capable of original breakthroughs.