The bottleneck for AI is not raw intelligence but understanding new context. This requires models that continuously learn from new data and interactions, moving beyond the static pre-train/fine-tune paradigm and deeply baking new information into the model weights.

The next major evolution in AI will be models that are personalized for specific users or companies and update their knowledge daily from interactions. This contrasts with current monolithic models like ChatGPT, which are static and must store irrelevant information for every user.

The popular conception of AGI as a pre-trained system that knows everything is flawed. A more realistic and powerful goal is an AI with a human-like ability for continual learning. This system wouldn't be deployed as a finished product, but as a 'super-intelligent 15-year-old' that learns and adapts to specific roles.

Google AI leader Jeff Dean highlighted "continual learning"—a model's ability to learn from new inputs post-training—as a key step toward AGI. That leaders are discussing it publicly suggests a breakthrough is near, which could rapidly accelerate AI capabilities and lead to a "fast takeoff" scenario.

The popular concept of AGI as a static, all-knowing entity is flawed. A more realistic and powerful model is one analogous to a 'super intelligent 15-year-old'—a system with a foundational capacity for rapid, continual learning. Deployment would involve this AI learning on the job, not arriving with complete knowledge.

Anthropic's pursuit of 'infinite context windows' could represent a practical breakthrough in continual learning. While debated by researchers, a model that can perpetually learn from its experiences within an ever-expanding context would, for all practical purposes, be a continually learning system, collapsing the functional distinction and moving closer to AGI.

Demis Hassabis argues that current LLMs are limited by their "goldfish brain"—they can't permanently learn from new interactions. He identifies solving this "continual learning" problem, where the model itself evolves over time, as one of the critical innovations needed to move from current systems to true AGI.

The key to a truly intelligent enterprise AI is not a static model, but one that uses reinforcement learning (RL) to continuously update its own weights overnight based on daily interactions, a concept known as 'continuous learning'.

A major flaw in current AI is that models are frozen after training and don't learn from new interactions. "Nested Learning," a new technique from Google, offers a path for models to continually update, mimicking a key aspect of human intelligence and overcoming this static limitation.