AI doesn't store data like a traditional database; it learns patterns and relationships, effectively compressing vast amounts of repetitive information. This is why a model trained on the entire internet can fit on a USB stick—it captures the essence and variations of concepts, not every single instance.

Simply using the most powerful model to generate synthetic data for a smaller model often fails. Effective distillation requires matching the 'teacher' model's token probabilities to the 'student' model's base architecture and training data, making it a complex research problem.

China is gaining an efficiency edge in AI by using "distillation"—training smaller, cheaper models from larger ones. This "train the trainer" approach is much faster and challenges the capital-intensive US strategy, highlighting how inefficient and "bloated" current Western foundational models are.

Instead of relying solely on massive, expensive, general-purpose LLMs, the trend is toward creating smaller, focused models trained on specific business data. These "niche" models are more cost-effective to run, less likely to hallucinate, and far more effective at performing specific, defined tasks for the enterprise.

The trend for language models is diverging: massive models in the cloud and smaller models (SLMs) at the edge. These SLMs, while lacking the broad knowledge of their larger counterparts, are highly effective when fine-tuned for specific domains and specialized data, making them ideal for device-level intelligence.

The public-facing models from major labs are likely efficient Mixture-of-Experts (MOE) versions distilled from much larger, private, and computationally expensive dense models. This means the model users interact with is a smaller, optimized copy, not the original frontier model.

Waymo uses a foundation model to create specialized, high-capacity "teacher" models (Driver, Simulator, Critic) offline. These teachers then distill their knowledge into smaller, efficient "student" models that can run in real-time on the vehicle, balancing massive computational power with on-device constraints.

Google's strategy involves creating both cutting-edge models (Pro/Ultra) and efficient ones (Flash). The key is using distillation to transfer capabilities from large models to smaller, faster versions, allowing them to serve a wide range of use cases from complex reasoning to everyday applications.

For low-latency applications, start with a small model to rapidly iterate on data quality. Then, use a large, high-quality model for optimal tuning with the cleaned data. Finally, distill the capabilities of this large, specialized model back into a small, fast model for production deployment.