Top AI lab leaders, including Demis Hassabis (Google DeepMind) and Dario Amodei (Anthropic), have publicly stated a desire to slow down AI development. They advocate for a collaborative, CERN-like model for AGI research but admit that intense, uncoordinated global competition currently makes such a pause impossible.
Viewing AGI development as a race with a winner-takes-all finish line is a risky assumption. It's more likely an ongoing competition where systems become progressively more advanced and diffused across applications, making the idea of a single "winner" misleading.
The idea of nations collectively creating policies to slow AI development for safety is naive. Game theory dictates that the immense competitive advantage of achieving AGI first will drive nations and companies to race ahead, making any global regulatory agreement effectively unenforceable.
Silicon Valley insiders, including former Google CEO Eric Schmidt, believe AI capable of improving itself without human instruction is just 2-4 years away. This shift in focus from the abstract concept of superintelligence to a specific research goal signals an imminent acceleration in AI capabilities and associated risks.
The hype around an imminent Artificial General Intelligence (AGI) event is fading among top AI practitioners. The consensus is shifting to a "Goldilocks scenario" where AI provides massive productivity gains as a synergistic tool, with true AGI still at least a decade away.
There's a stark contrast in AGI timeline predictions. Newcomers and enthusiasts often predict AGI within months or a few years. However, the field's most influential figures, like Ilya Sutskever and Andrej Karpathy, are now signaling that true AGI is likely decades away, suggesting the current paradigm has limitations.
A fundamental tension within OpenAI's board was the catch-22 of safety. While some advocated for slowing down, others argued that being too cautious would allow a less scrupulous competitor to achieve AGI first, creating an even greater safety risk for humanity. This paradox fueled internal conflict and justified a rapid development pace.
Companies like OpenAI and Anthropic are not just building better models; their strategic goal is an "automated AI researcher." The ability for an AI to accelerate its own development is viewed as the key to getting so far ahead that no competitor can catch up.
Demis Hassabis, CEO of Google DeepMind, warns that the societal transition to AGI will be immensely disruptive, happening at a scale and speed ten times greater than the Industrial Revolution. This suggests that historical parallels are inadequate for planning and preparation.
Anthropic CEO Dario Amodei's writing proposes using an AI advantage to 'make China an offer they can't refuse,' forcing them to abandon competition with democracies. The host argues this is an extremely reckless position that fuels an arms race dynamic, especially when other leaders like Google's Demis Hassabis consistently call for international collaboration.
Individual teams within major AI labs often act responsibly within their constrained roles. However, the overall competitive dynamic and lack of coordination between companies leads to a globally reckless situation, where risks are accepted that no single, rational entity would endorse.