Get your free personalized podcast brief
We scan new podcasts and send you the top 5 insights daily.
David Weisberg observes that "recursive self-improvement"—an AI's ability to improve itself—was considered the definition of AGI just two years ago. As models approach this capability, the goalposts for AGI are moving, suggesting we are entering an era that was recently defined as the technological singularity.
Related Insights
Today's AI models have surpassed the definition of Artificial General Intelligence (AGI) that was commonly accepted by AI researchers just over a decade ago. The debate continues because the goalposts for what constitutes "true" AGI have been moved.
The vague concept of AGI is being replaced by Recursive Self-Improvement (RSI)—AI models creating their own successors. This is seen as a more specific and potentially nearer-term threshold that could trigger an uncontrolled explosion in AI progress, moving humans "out of the loop entirely."
Mustafa Suleiman offers clear definitions: AGI is human parity on most tasks. Superintelligence exceeds human performance and discovers new knowledge. The Singularity is the sci-fi point where a superintelligence can recursively self-improve. This clarifies the ladder of AI progression beyond generic terms.
Silicon Valley insiders, including former Google CEO Eric Schmidt, believe AI capable of improving itself without human instruction is just 2-4 years away. This shift in focus from the abstract concept of superintelligence to a specific research goal signals an imminent acceleration in AI capabilities and associated risks.
The pursuit of AGI may mirror the history of the Turing Test. Once ChatGPT clearly passed the test, the milestone was dismissed as unimportant. Similarly, as AI achieves what we now call AGI, society will likely move the goalposts and decide our original definition was never the true measure of intelligence.
The definition of AGI is a moving goalpost. Scott Wu argues that today's AI meets the standards that would have been considered AGI a decade ago. As technology automates tasks, human work simply moves to a higher level of abstraction, making percentage-based definitions of AGI flawed.
The debate over AGI is skewed because the goalposts have continuously moved. According to Cerebras CEO Andrew Feldman, if we apply any standard definition of Artificial General Intelligence from a decade or two ago, such as the Turing Test, current AI models have already blown past it. The achievement is historical; our expectations are what keep changing.
Even as AI models surpass technical AGI benchmarks, the host argues people will keep moving the goalposts. The true, socially accepted definition of AGI will be its "feel"—its ability to generalize and execute complex, nuanced tasks with minimal instruction, like a human.
Top AI labs see the race ending not with an IPO, but with "recursive self-improvement"—the moment a model can code its own next version, causing progress to "go vertical." One lab leader believes this will happen by 2028. The strategy is to maintain a lead for just a few more years to win the race permanently.
Sequoia highlights the "AI effect": once an AI capability becomes mainstream, we stop calling it AI and give it a specific name, thereby moving the goalposts for "true" AI. This historical pattern of downplaying achievements is a key reason they are explicitly declaring the arrival of AGI.
