Research with long timelines (e.g., a "2063 scenario") is still worth pursuing, as these technical plans can be compressed into a short period by future AI assistants. Seeding these directions now raises the "waterline of understanding" for future AI-accelerated alignment efforts, making them viable even on shorter timelines.
Related Insights
Emmett Shear reframes AI alignment away from a one-time problem to be solved. Instead, he presents it as an ongoing, living process of recalibration and learning, much like how human families or societies maintain cohesion. This challenges the common 'lock in values' approach in AI safety.
The development of superintelligence is unique because the first major alignment failure will be the last. Unlike other fields of science where failure leads to learning, an unaligned superintelligence would eliminate humanity, precluding any opportunity to try again.
Current AI alignment focuses on how AI should treat humans. A more stable paradigm is "bidirectional alignment," which also asks what moral obligations humans have toward potentially conscious AIs. Neglecting this could create AIs that rationally see humans as a threat due to perceived mistreatment.
Ryan Kidd of MATS, a major AI safety talent pipeline, uses a 2033 median AGI timeline from prediction markets like Metaculous for strategic planning. This provides a concrete, data-driven anchor for how a key organization in the space views timelines, while still preparing for shorter, more dangerous scenarios.
Julian Schrittwieser, a key researcher from Anthropic and formerly Google DeepMind, forecasts that extrapolating current AI progress suggests models will achieve full-day autonomy and match human experts across many industries by mid-2026. This timeline is much shorter than many anticipate.
OpenAI announced goals for an AI research intern by 2026 and a fully autonomous researcher by 2028. This isn't just a scientific pursuit; it's a core business strategy to exponentially accelerate AI discovery by automating innovation itself, which they plan to sell as a high-priced agent.
There's a stark contrast in AGI timeline predictions. Newcomers and enthusiasts often predict AGI within months or a few years. However, the field's most influential figures, like Ilya Sutskever and Andrej Karpathy, are now signaling that true AGI is likely decades away, suggesting the current paradigm has limitations.
The ultimate goal for leading labs isn't just creating AGI, but automating the process of AI research itself. By replacing human researchers with millions of "AI researchers," they aim to trigger a "fast takeoff" or recursive self-improvement. This makes automating high-level programming a key strategic milestone.
Sam Altman's goal of an "automated AI research intern" by 2026 and a full "researcher" by 2028 is not about simple task automation. It is a direct push toward creating recursively self-improving systems—AI that can discover new methods to improve AI models, aiming for an "intelligence explosion."
Treating AI alignment as a one-time problem to be solved is a fundamental error. True alignment, like in human relationships, is a dynamic, ongoing process of learning and renegotiation. The goal isn't to reach a fixed state but to build systems capable of participating in this continuous process of re-knitting the social fabric.
