OpenAI co-founder Ilya Sutskever suggests the path to AGI is not creating a pre-trained, all-knowing model, but an AI that can learn any task as effectively as a human. This reframes the challenge from knowledge transfer to creating a universal learning algorithm, impacting how such systems would be deployed.

Even with vast training data, current AI models are far less sample-efficient than humans. This limits their ability to adapt and learn new skills on the fly. They resemble a perpetual new hire who can access information but lacks the deep, instinctual learning that comes from experience and weight updates.

The popular conception of AGI as a pre-trained system that knows everything is flawed. A more realistic and powerful goal is an AI with a human-like ability for continual learning. This system wouldn't be deployed as a finished product, but as a 'super-intelligent 15-year-old' that learns and adapts to specific roles.

The popular concept of AGI as a static, all-knowing entity is flawed. A more realistic and powerful model is one analogous to a 'super intelligent 15-year-old'—a system with a foundational capacity for rapid, continual learning. Deployment would involve this AI learning on the job, not arriving with complete knowledge.

The essence of Sutton's "Bitter Lesson" is a warning against the temptation to embed human knowledge into AI systems. Instead, AI progress is driven by general methods like search and learning that scale with computation, a lesson learned over decades of research.

Rich Sutton posits that the need to specify "continual learning" is a recent, strange development. Historically, all learning was assumed to be ongoing. The current AI paradigm of training static models is an aberration from this natural, common-sense view of intelligence.

The current focus on pre-training AI with specific tool fluencies overlooks the crucial need for on-the-job, context-specific learning. Humans excel because they don't need pre-rehearsal for every task. This gap indicates AGI is further away than some believe, as true intelligence requires self-directed, continuous learning in novel environments.

The process of training an AI—starting from ignorance and learning via trial-and-error with reward signals—is a powerful analogy for conscious learning in animals. This iterative, goal-directed process may be more relevant to the emergence of subjective experience than the final, deployed model's inference tasks.

Just as crawling is a vital developmental step for babies even though adults don't crawl, some learning processes that AI can automate might be essential for cognitive development. We shouldn't skip steps without understanding their underlying neurological purpose.