While more data and compute yield linear improvements, true step-function advances in AI come from unpredictable algorithmic breakthroughs like Transformers. These creative ideas are the most difficult to innovate on and represent the highest-leverage, yet riskiest, area for investment and research focus.

A 10x increase in compute may only yield a one-tier improvement in model performance. This appears inefficient but can be the difference between a useless "6-year-old" intelligence and a highly valuable "16-year-old" intelligence, unlocking entirely new economic applications.

The success of neural networks on problems like Go and protein folding, long considered intractable NP-hard problems, is profound. It suggests our formal understanding of computational hardness, which focuses on worst-case scenarios, may be an incomplete model for how to find useful, approximate solutions in practice.

Breakthroughs like neural network "pruning" can reduce model size by 90% without losing accuracy, offering a 10x reduction in inference costs. This highlights that algorithmic innovation, not just acquiring more hardware, will be a key competitive vector in the AI race, enabling more output with less energy.

The history of AI, such as the 2012 AlexNet breakthrough, demonstrates that scaling compute and data on simpler, older algorithms often yields greater advances than designing intricate new ones. This "bitter lesson" suggests prioritizing scalability over algorithmic complexity for future progress.

Major AI breakthroughs, like OpenAI's solving of a 200-year-old math problem, are often misunderstood as magical insight. In reality, they represent a massive application of computational leverage—equivalent to tens of thousands of human work-years. AI's value is as an engine for brute-force problem-solving, not a mystical god.

While training AI is vastly less data-efficient than training a human, it remains a winning economic strategy. Unlike humans, AI training can be massively parallelized, and the resulting skills can be amortized across billions of simultaneous user sessions, making the inefficient process highly profitable and scalable.

The common goal of increasing AI model efficiency could have a paradoxical outcome. If AI performance becomes radically cheaper ("too cheap to meter"), it could devalue the massive investments in compute and data center infrastructure, creating a financial crisis for the very companies that enabled the boom.

A radical improvement in compute efficiency won't just lower costs; it will trigger Jevons' paradox, where consumption increases by more than the price drops. Making AI compute 1000x cheaper will unlock currently unimaginable applications, creating a market far larger than linear projections and potentially the largest in human history.