For vertical AI applications, foundation models are now sufficiently intelligent. The primary challenge is no longer model capability but building the surrounding software infrastructure—tools, UIs, and workflows—that lets models perform useful work reliably and trustworthily.

Overly structured, workflow-based systems that work with today's models will become bottlenecks tomorrow. Engineers must be prepared to shed abstractions and rebuild simpler, more general systems to capture the gains from exponentially improving models.

Classic software engineering warns against full rewrites due to risk and time ("second-system syndrome"). However, AI's ability to rebuild an entire product in days, not years, makes rewriting a powerful and low-cost tool for correcting over-complicated early versions or flawed core assumptions.

The era of guaranteed progress by simply scaling up compute and data for pre-training is ending. With massive compute now available, the bottleneck is no longer resources but fundamental ideas. The AI field is re-entering a period where novel research, not just scaling existing recipes, will drive the next breakthroughs.

Google's new state-of-the-art Deep Research agents are still powered by the older Gemini 3.1 Pro model. Their significant performance improvements come entirely from 'harness upgrades' and additional inference techniques. This demonstrates that the systems, tools, and processes surrounding a model are now a primary driver of capability, not just the raw power of the base model itself.

The dominant AI development method involves creating a thin scaffold for a task, capturing errors, and then letting the model rewrite its own code to correct those mistakes. This "correction by correction" loop allows AI systems to improve their capabilities at an astonishingly rapid pace.

OpenAI's model development isn't about isolated releases. A new pre-trained base model like 'Spud' acts as a new foundation. It allows two years' worth of accumulated but previously unrealized research in areas like reinforcement learning and fine-tuning to finally come to fruition, creating a step-change in capability.

Building on AI requires creating custom infrastructure to fill performance gaps. As underlying models improve, founders must be prepared to delete this now-redundant code and upgrade their product vision to tackle the next set of challenges at the new frontier. This cycle of building and deleting is key to staying innovative.

The rapid pace of AI paradigm shifts—from simple token-in/token-out models to complex agentic systems—forces a complete infrastructure rewrite every 12 to 18 months. Google's lesson for large organizations is to invest in standardized platforms to avoid having every team reinvent the wheel and fall behind.