The bottleneck for AI is not raw intelligence but understanding new context. This requires models that continuously learn from new data and interactions, moving beyond the static pre-train/fine-tune paradigm and deeply baking new information into the model weights.

The next major evolution in AI will be models that are personalized for specific users or companies and update their knowledge daily from interactions. This contrasts with current monolithic models like ChatGPT, which are static and must store irrelevant information for every user.

The popular conception of AGI as a pre-trained system that knows everything is flawed. A more realistic and powerful goal is an AI with a human-like ability for continual learning. This system wouldn't be deployed as a finished product, but as a 'super-intelligent 15-year-old' that learns and adapts to specific roles.

Karpathy identifies a key missing piece for continual learning in AI: an equivalent to sleep. Humans seem to use sleep to distill the day's experiences (their "context window") into the compressed weights of the brain. LLMs lack this distillation phase, forcing them to restart from a fixed state in every new session.

AI agents like OpenClaw learn via "skills"—pre-written text instructions. While functional, this method is described as "janky" and a workaround. It exposes a core weakness of current AI: the lack of true continual learning. This limitation is so profound that new startups are rethinking AI architecture from scratch to solve it.

Despite marketing claims, current AI agents cannot truly learn or improve over time like a human employee. They operate by consulting static knowledge bases, not by gaining experience. This "narrative gap" between public perception and actual capability is a major industry challenge.

A genuinely continual learner doesn't have separate training and testing phases. Instead, its life is a continuous process divided into two modes: an 'active' phase of interacting with new data and an 'offline' sleep phase for memory consolidation and self-improvement.

Demis Hassabis argues that current LLMs are limited by their "goldfish brain"—they can't permanently learn from new interactions. He identifies solving this "continual learning" problem, where the model itself evolves over time, as one of the critical innovations needed to move from current systems to true AGI.

The key to a truly intelligent enterprise AI is not a static model, but one that uses reinforcement learning (RL) to continuously update its own weights overnight based on daily interactions, a concept known as 'continuous learning'.