A fundamental, unsolved problem in continual learning is teaching AI models how to distinguish between legitimate new information and malicious, fake data fed by users. This represents a critical security and reliability challenge before the technology can be widely and safely deployed.

Cohere's Chief AI Officer, Joelle Pineau, finds the concept of continual learning problematic because the research community lacks a universally agreed-upon problem definition, making it difficult to measure progress, unlike more standardized research areas like AI memory.

AI agents like OpenClaw learn via "skills"—pre-written text instructions. While functional, this method is described as "janky" and a workaround. It exposes a core weakness of current AI: the lack of true continual learning. This limitation is so profound that new startups are rethinking AI architecture from scratch to solve it.

Many AI projects fail to reach production because of reliability issues. The vision for continual learning is to deploy agents that are 'good enough,' then use RL to correct behavior based on real-world errors, much like training a human. This solves the final-mile reliability problem and could unlock a vast market.

Adaption.AI is bucking the trend of building larger static models to focus on continual learning. Their core mission is to 'eliminate prompt engineering,' viewing it as a crutch that signifies a model's failure to truly adapt and learn from user interaction in real-time.

Dario Amodei argues that the current AI paradigm—combining broad generalization from pre-training/RL with vast in-context learning—is likely powerful enough to create trillions of dollars in value. He posits that solving "continual learning," where a model learns permanently on the job, is a desirable but potentially non-essential next step.

Demis Hassabis argues that current LLMs are limited by their "goldfish brain"—they can't permanently learn from new interactions. He identifies solving this "continual learning" problem, where the model itself evolves over time, as one of the critical innovations needed to move from current systems to true AGI.

The key to continual learning is not just a longer context window, but a new architecture with a spectrum of memory types. "Nested learning" proposes a model with different layers that update at different frequencies—from transient working memory to persistent core knowledge—mimicking how humans learn without catastrophic forgetting.

A major flaw in current AI is that models are frozen after training and don't learn from new interactions. "Nested Learning," a new technique from Google, offers a path for models to continually update, mimicking a key aspect of human intelligence and overcoming this static limitation.