While more data and compute yield linear improvements, true step-function advances in AI come from unpredictable algorithmic breakthroughs like Transformers. These creative ideas are the most difficult to innovate on and represent the highest-leverage, yet riskiest, area for investment and research focus.

With industry dominating large-scale compute, academia's function is no longer to train the biggest models. Instead, its value lies in pursuing unconventional, high-risk research in areas like new algorithms, architectures, and theoretical underpinnings that commercial labs, focused on scaling, might overlook.

The era of advancing AI simply by scaling pre-training is ending due to data limits. The field is re-entering a research-heavy phase focused on novel, more efficient training paradigms beyond just adding more compute to existing recipes. The bottleneck is shifting from resources back to ideas.

A "software-only singularity," where AI recursively improves itself, is unlikely. Progress is fundamentally tied to large-scale, costly physical experiments (i.e., compute). The massive spending on experimental compute over pure researcher salaries indicates that physical experimentation, not just algorithms, remains the primary driver of breakthroughs.

With industry dominating large-scale model training, academic labs can no longer compete on compute. Their new strategic advantage lies in pursuing unconventional, high-risk ideas, new algorithms, and theoretical underpinnings that large commercial labs might overlook.

The history of AI, such as the 2012 AlexNet breakthrough, demonstrates that scaling compute and data on simpler, older algorithms often yields greater advances than designing intricate new ones. This "bitter lesson" suggests prioritizing scalability over algorithmic complexity for future progress.

A nation's advantage is its "intelligent capital stock": its total GPU compute power multiplied by the quality of its AI models. This explains the US restricting GPU sales to China, which counters by excelling in open-source models to close the gap.

For years, access to compute was the primary bottleneck in AI development. Now, as public web data is largely exhausted, the limiting factor is access to high-quality, proprietary data from enterprises and human experts. This shifts the focus from building massive infrastructure to forming data partnerships and expertise.

Small firms can outmaneuver large corporations in the AI era by embracing rapid, low-cost experimentation. While enterprises spend millions on specialized PhDs for single use cases, agile companies constantly test new models, learn from failures, and deploy what works to dominate their market.