The 2012 breakthrough that ignited the modern AI era used the ImageNet dataset, a novel neural network, and only two NVIDIA gaming GPUs. This demonstrates that foundational progress can stem from clever architecture and the right data, not just massive initial compute power, a lesson often lost in today's scale-focused environment.

AI development history shows that complex, hard-coded approaches to intelligence are often superseded by more general, simpler methods that scale more effectively. This "bitter lesson" warns against building brittle solutions that will become obsolete as core models improve.

The history of AI, such as the 2012 AlexNet breakthrough, demonstrates that scaling compute and data on simpler, older algorithms often yields greater advances than designing intricate new ones. This "bitter lesson" suggests prioritizing scalability over algorithmic complexity for future progress.

Model performance isn't just about architecture; it's also about compute budget. A less sophisticated AI model, if allowed to run for longer or iterate more times, can often match the output of a state-of-the-art model. This suggests access to cheap energy could be a greater advantage than access to the best chips.

The 'bigger is better' narrative is breaking down. For well-defined, structured tasks like coding and math, small models (e.g., 3 billion parameters) are now matching the performance of frontier models. This enables powerful, specialized AI to run on modest local hardware.

Despite billions spent on AI hype, established and simpler algorithms continue to deliver trillion-dollar returns on investment. The focus on complex, cutting-edge AI often overshadows the immense and ongoing value derived from older, more straightforward mathematical and statistical models that are less costly and more reliable.

Microsoft's research found that training smaller models on high-quality, synthetic, and carefully filtered data produces better results than training larger models on unfiltered web data. Data quality and curation, not just model size, are the new drivers of performance.

The AI model's advantage came from its ability to solve the numbers problem: objectively assessing team strength based on squad value and form. The human journalist's correct predictions, however, came from reading intangible factors like team morale, a coach's influence, or player behavior under pressure—elements a Poisson model cannot see. The true takeaway is that AI and human intuition are suited for different parts of the same problem.

The focus on benchmark scores for frontier models is misplaced for most practical use cases. Many applications, especially in physical and embedded AI, rely on smaller, specialized models. The small percentage point differences on abstract benchmarks have little bearing on solving a specific business problem effectively.