As AI models achieve previously defined benchmarks for intelligence (e.g., reasoning), their failure to generate transformative economic value reveals those benchmarks were insufficient. This justifies 'shifting the goalposts' for AGI. It is a rational response to realizing our understanding of intelligence was too narrow. Progress in impressiveness doesn't equate to progress in usefulness.
Related Insights
The most immediate AI milestone is not singularity, but "Economic AGI," where AI can perform most virtual knowledge work better than humans. This threshold, predicted to arrive within 12-18 months, will trigger massive societal and economic shifts long before a "Terminator"-style superintelligence becomes a reality.
A consortium including leaders from Google and DeepMind has defined AGI as matching the cognitive versatility of a "well-educated adult" across 10 domains. This new framework moves beyond abstract debate, showing a concrete 30-point leap in AGI score from GPT-4 (27%) to a projected GPT-5 (57%).
AI intelligence shouldn't be measured with a single metric like IQ. AIs exhibit "jagged intelligence," being superhuman in specific domains (e.g., mastering 200 languages) while simultaneously lacking basic capabilities like long-term planning, making them fundamentally unlike human minds.
The argument that AI adoption is slow due to normal tech diffusion is flawed. If AI models possessed true human-equivalent capabilities, they would be adopted faster than human employees because they could onboard instantly and eliminate hiring risks. The current lack of widespread economic value is direct evidence that today's AI models are not yet capable enough for broad deployment.
OpenAI's CEO believes the term "AGI" is ill-defined and its milestone may have passed without fanfare. He proposes focusing on "superintelligence" instead, defining it as an AI that can outperform the best human at complex roles like CEO or president, creating a clearer, more impactful threshold.
The definition of AGI is a moving goalpost. Scott Wu argues that today's AI meets the standards that would have been considered AGI a decade ago. As technology automates tasks, human work simply moves to a higher level of abstraction, making percentage-based definitions of AGI flawed.
The disconnect between AI's superhuman benchmark scores and its limited economic impact exists because many benchmarks test esoteric problems. The Arc AGI prize instead focuses on tasks that are easy for humans, testing an AI's ability to learn new concepts from few examples—a better proxy for general, applicable intelligence.
The argument is that "economic diffusion lag" is an excuse for AI's current limitations. If AI models were truly as capable as human employees, they would integrate into companies instantly—far faster than human hiring. The slow rollout proves they still lack core, necessary skills for broad economic value.
The slow adoption of AI isn't due to a natural 'diffusion lag' but is evidence that models still lack core competencies for broad economic value. If AI were as capable as skilled humans, it would integrate into businesses almost instantly.
OpenAI's new GDP-val benchmark evaluates models on complex, real-world knowledge work tasks, not abstract IQ tests. This pivot signifies that the true measure of AI progress is now its ability to perform economically valuable human jobs, making performance metrics directly comparable to professional output.
