Accusations that Chinese labs cheat by copying US models are misleading. The practice, known as distillation, is common across the industry (including by Elon Musk's xAI) and academia. Now, with Chinese labs dominating open source, American startups are increasingly building on top of Chinese models.

As more of the public internet and code repositories are generated by LLMs, any new model trained on this public data is, in effect, being 'distilled' from other models. This complicates accusations of direct distillation and blurs the line for what constitutes original training data.

Large, centralized AI models are vulnerable to 'distillation attacks,' where a smaller model can be trained cheaply by querying the larger one. This technical reality, combined with the moral hypocrisy of creators restricting copying after scraping the internet, strongly suggests a future dominated by decentralized, open-source models.

As more of the internet and code repositories are generated by leading AI models, any new model trained on this public data inadvertently "distills" the knowledge and quirks of those proprietary systems. This blurs the line between original training and outright copying.

Advanced model training is not just about scraping the web. It's a multi-stage process that starts with massive web data, is refined by human-created examples and ratings (SFT), and is then scaled using reinforcement learning on data generated by the model itself. This synthetic data loop is now a critical component.

The public-facing models from major labs are likely efficient Mixture-of-Experts (MOE) versions distilled from much larger, private, and computationally expensive dense models. This means the model users interact with is a smaller, optimized copy, not the original frontier model.

A new form of analysis compares the semantic similarities (e.g., diction, phrasing) of outputs from different AI models. This technique is being used to create 'fingerprints' that can suggest if one model was illicitly 'distilled' or trained on the outputs of another, a key concern in the AI arms race.

As developers increasingly use AI coding assistants like Claude Code, they flood public repositories like GitHub with high-quality, AI-generated outputs. This effectively turns the internet into a massive, unavoidable training dataset for competing models, making it difficult to police "distillation" as a violation of terms.

New open-weight models like Inkling are not entirely 'pure'; they use 'distillation light' from other open models (e.g., Kimi). Since those models may be distilled from closed-source giants like OpenAI, it creates a multi-layered dependency chain where traits and biases are passed down, blurring the lines between truly independent and derivative models.