AI adoption isn't linear. A small, 1% improvement in model capability can be the critical step that clears a usability hurdle, transforming a "toy" into a production-ready tool. This creates sudden, discontinuous leaps in market adoption that are hard to predict from capability trend lines alone.

The dramatic improvements from GPT-2 to GPT-4 were driven by a simple law: bigger models and more training data yielded better results. This trend has stopped. Recent attempts to scale even larger models have produced only marginal gains, forcing the industry into more complex, narrow optimizations instead of giant leaps.

The sudden arrival of powerful AI like GPT-3 was a non-repeatable event: training on the entire internet and all existing books. With this data now fully "eaten," future advancements will feel more incremental, relying on the slower process of generating new, high-quality expert data.

The future of AI is hard to predict because increasing a model's scale often produces 'emergent properties'—new capabilities that were not designed or anticipated. This means even experts are often surprised by what new, larger models can do, making the development path non-linear.

Over two-thirds of reasoning models' performance gains came from massively increasing their 'thinking time' (inference scaling). This was a one-time jump from a zero baseline. Further gains are prohibitively expensive due to compute limitations, meaning this is not a repeatable source of progress.

A key surprise in AI development was the non-linear impact of scale. Sebastian Thrun noted that while AI trained on millions of documents is 'fine,' training it on hundreds of billions creates an 'unbelievably smart' system, shocking even its creators and demonstrating data volume as a primary driver of breakthroughs.

AI labs like Anthropic find that mid-tier models can be trained with reinforcement learning to outperform their largest, most expensive models in just a few months, accelerating the pace of capability improvements.

While AI progress is marketed in revolutionary "step-changes" (e.g., GPT-3 to GPT-4), the underlying reality is more like compounding interest. A continuous stream of small, incremental improvements are accumulating, and their combined effect is what creates the feeling of an exponential leap in capability over time.

Dario Amodei stands by his 2017 "big blob of compute" hypothesis. He argues that AI breakthroughs are driven by scaling a few core elements—compute, data, training time, and a scalable objective—rather than clever algorithmic tricks, a view similar to Rich Sutton's "Bitter Lesson."