Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

The history of AI development is framed as a pattern of underestimation. Skeptics repeatedly set capability benchmarks they believe AI will never reach (e.g., solving high-level math problems). Once AI achieves them, the goalposts are simply moved, ignoring the accelerating trend of progress.

Related Insights

As soon as OpenAI's Astra model nearly 'solved' the ARC AGI 3 benchmark, its creator immediately 'moved the goalposts' by announcing the next version will focus on 'open-ended invention.' This shows how the very definition of AGI is a moving target, constantly being redefined by technological breakthroughs.

A 2022 study by the Forecasting Research Institute has been reviewed, revealing that top forecasters and AI experts significantly underestimated AI advancements. They assigned single-digit odds to breakthroughs that occurred within two years, proving we are consistently behind the curve in our predictions.

The same group of AI alarmists have a track record of failed predictions, from GPT-2 being 'too dangerous to release' to massive job losses that never materialized. As each dire prediction is refuted by reality, the doomer narrative simply moves to the next hypothetical threat without acknowledging past errors.

While AI progress is marketed in revolutionary "step-changes" (e.g., GPT-3 to GPT-4), the underlying reality is more like compounding interest. A continuous stream of small, incremental improvements are accumulating, and their combined effect is what creates the feeling of an exponential leap in capability over time.

For years, critics have claimed deep learning is about to plateau, citing specific limitations like causal or common-sense reasoning. These "walls" have been consistently overcome within a few years, suggesting current predictions of a plateau are based on a historically unreliable intuition.

The pursuit of AGI may mirror the history of the Turing Test. Once ChatGPT clearly passed the test, the milestone was dismissed as unimportant. Similarly, as AI achieves what we now call AGI, society will likely move the goalposts and decide our original definition was never the true measure of intelligence.

As AI achieves impressive milestones, like assisting in creating a cancer vaccine, the public conversation immediately discounts the achievement. The goalposts shift from "AI helped solve a problem" to demanding a fully autonomous, one-shot solution. This pattern of escalating expectations obscures the real, incremental progress being made.

The belief that AI progress will be slow often stems from a strong prior that 'things are just always hard and slow.' This 'bottleneck objection' leads skeptics to assume unforeseen drag factors will always emerge, causing them to dismiss detailed scenarios for rapid acceleration without engaging with the specifics.

Criticizing AI developers for being a few months off on predictions is a distraction. The underlying trend is one of exponential growth. Like criticizing Elon Musk's Mars timeline while ignoring his historic rocket launches, it's a failure to grasp the scale and direction of the technological shift that is already happening.

The discourse around AGI is caught in a paradox. Either it is already emerging, in which case it's less a cataclysmic event and more an incremental software improvement, or it remains a perpetually receding future goal. This captures the tension between the hype of superhuman intelligence and the reality of software development.