The AI development cycle of experimentation and bottleneck-solving is already a form of recursive self-improvement. Kyle Corbitt argues this loop is currently constrained by human intelligence. Once AIs become better at directing this process, progress will accelerate rapidly.

The most transformative aspect of AI may be its ability to automate its own research and development. This creates a recursive improvement cycle—an "intelligence explosion"—where progress accelerates exponentially, compressing decades of innovation into a much shorter period.

A "software-only singularity," where AI recursively improves itself, is unlikely. Progress is fundamentally tied to large-scale, costly physical experiments (i.e., compute). The massive spending on experimental compute over pure researcher salaries indicates that physical experimentation, not just algorithms, remains the primary driver of breakthroughs.

A key bottleneck in predicting fast intelligence explosions is the difficulty of measuring an AI's "research taste"—its ability to design good experiments and set research direction. This skill, more than coding, may be the primary driver of a rapid takeoff. It is a critical parameter that is currently not well-studied or benchmarked.

Beyond just coding, improving AI models requires subtle skills like designing effective reinforcement learning environments or managing human expert feedback. Newman questions how close we are to recursive self-improvement by asking if AIs can automate these tasks, which rely on nuanced "taste and judgment" rather than just raw computational ability.

Even if AI perfects software engineering, automating AI R&D will be limited by non-coding tasks, as AI companies aren't just software engineers. Furthermore, AI assistance might only be enough to maintain the current rate of progress as 'low-hanging fruit' disappears, rather than accelerate it.

Rohin Shah predicts a gradual, not abrupt, start to an intelligence explosion. It will be triggered when automated AI R&D becomes cheaper than human researchers, not when it's vastly more capable. The first automated researchers might be less insightful but use massive, expensive compute to brute-force problems.

The ultimate goal for leading labs isn't just creating AGI, but automating the process of AI research itself. By replacing human researchers with millions of "AI researchers," they aim to trigger a "fast takeoff" or recursive self-improvement. This makes automating high-level programming a key strategic milestone.

A key failure mode for using AI to solve AI safety is an 'unlucky' development path where models become superhuman at accelerating AI R&D before becoming proficient at safety research or other defensive tasks. This could create a period where we know an intelligence explosion is imminent but are powerless to use the precursor AIs to prepare for it.