The bottleneck for AI is not raw intelligence but understanding new context. This requires models that continuously learn from new data and interactions, moving beyond the static pre-train/fine-tune paradigm and deeply baking new information into the model weights.

The popular conception of AGI as a pre-trained system that knows everything is flawed. A more realistic and powerful goal is an AI with a human-like ability for continual learning. This system wouldn't be deployed as a finished product, but as a 'super-intelligent 15-year-old' that learns and adapts to specific roles.

Rich Sutton argues that LLM-based assistants are not "experiential learners." Despite in-context learning, their fundamental weights never change after deployment. This prevents them from truly adapting, forming new concepts, or updating their core understanding based on user interactions.

The AI community disagrees on how models should learn continuously. One camp favors updating model weights directly, while the 'systems' camp prefers storing memories in external databases for better control. The two sides are philosophically opposed, with enterprises strongly preferring the systems approach for its security and debuggability.

Sutton contends that focusing on supervised learning is a mistake when trying to build AGI. Animals, including humans, do not learn core skills like walking or seeing via labeled examples. True intelligence is rooted in experiential, goal-driven learning, making school-style supervision an irrelevant model for core AI.

A genuinely continual learner doesn't have separate training and testing phases. Instead, its life is a continuous process divided into two modes: an 'active' phase of interacting with new data and an 'offline' sleep phase for memory consolidation and self-improvement.

The key to a truly intelligent enterprise AI is not a static model, but one that uses reinforcement learning (RL) to continuously update its own weights overnight based on daily interactions, a concept known as 'continuous learning'.

A major flaw in current AI is that models are frozen after training and don't learn from new interactions. "Nested Learning," a new technique from Google, offers a path for models to continually update, mimicking a key aspect of human intelligence and overcoming this static limitation.

Current AI models are like interns: they execute tasks but don't learn from experience and effectively reset daily. True "continual learning" would allow AI to build on its experiences, transforming it from a temporary helper into a fully integrated, improving "employee."