Dario Amodei argues that the current AI paradigm—combining broad generalization from pre-training/RL with vast in-context learning—is likely powerful enough to create trillions of dollars in value. He posits that solving "continual learning," where a model learns permanently on the job, is a desirable but potentially non-essential next step.
Related Insights
Dario Amodei suggests that the massive data requirement for AI pre-training is not a flaw but a different paradigm. It is analogous to the long process of human evolution setting up our brain's priors, not just an individual's lifetime of learning, which explains its sample inefficiency.
The popular conception of AGI as a pre-trained system that knows everything is flawed. A more realistic and powerful goal is an AI with a human-like ability for continual learning. This system wouldn't be deployed as a finished product, but as a 'super-intelligent 15-year-old' that learns and adapts to specific roles.
The popular concept of AGI as a static, all-knowing entity is flawed. A more realistic and powerful model is one analogous to a 'super intelligent 15-year-old'—a system with a foundational capacity for rapid, continual learning. Deployment would involve this AI learning on the job, not arriving with complete knowledge.
Dario Amodei views the distinction between RL and pre-training scaling as a red herring. He argues that, just like early language models needed broad internet-scale data to generalize (GPT-2 vs. GPT-1), RL needs to move beyond narrow tasks to a wide variety of environments to achieve true generalization.
While desirable for adaptability, creating models that learn continuously risks a winner-take-all dynamic where one company's model becomes uncatchably superior. This also represents a risky 'depth-first search' toward AGI, prematurely committing to the current transformer paradigm without exploring safer alternatives.
Many AI projects fail to reach production because of reliability issues. The vision for continual learning is to deploy agents that are 'good enough,' then use RL to correct behavior based on real-world errors, much like training a human. This solves the final-mile reliability problem and could unlock a vast market.
Dario Amodei stands by his 2017 "big blob of compute" hypothesis. He argues that AI breakthroughs are driven by scaling a few core elements—compute, data, training time, and a scalable objective—rather than clever algorithmic tricks, a view similar to Rich Sutton's "Bitter Lesson."
The current focus on pre-training AI with specific tool fluencies overlooks the crucial need for on-the-job, context-specific learning. Humans excel because they don't need pre-rehearsal for every task. This gap indicates AGI is further away than some believe, as true intelligence requires self-directed, continuous learning in novel environments.
Demis Hassabis argues that current LLMs are limited by their "goldfish brain"—they can't permanently learn from new interactions. He identifies solving this "continual learning" problem, where the model itself evolves over time, as one of the critical innovations needed to move from current systems to true AGI.
The perceived need for a new "continual learning" architecture is overstated. Current models can already achieve this functionally by building their own tools and apps based on new information. This reframes the challenge from a fundamental research problem to a practical prompt engineering and application design issue.
