The bottleneck for AI is not raw intelligence but understanding new context. This requires models that continuously learn from new data and interactions, moving beyond the static pre-train/fine-tune paradigm and deeply baking new information into the model weights.

Inspired by human dreaming as a memory reconsolidation process, Anthropic has its AI agents use downtime to "dream." During this background process, the agent reviews its memories, identifies and prunes contradictions, and cleans up the information to improve the coherence and utility of its long-term memory.

Inspired by human sleep, AI models can enter an offline mode. During this 'sleep,' they consolidate new knowledge from fast-updating layers into slow-updating ones via distillation. They also 'dream' by generating synthetic data from recent experiences to form new abstractions and connections.

Karpathy identifies a key missing piece for continual learning in AI: an equivalent to sleep. Humans seem to use sleep to distill the day's experiences (their "context window") into the compressed weights of the brain. LLMs lack this distillation phase, forcing them to restart from a fixed state in every new session.

The 'dreaming' phase in continual learning isn't just for memory consolidation. It serves to actively find connections between concepts that seem unrelated based on recent experiences. This process allows the model to form new, higher-level abstractions and insights, mirroring a key function of human dreaming.

To manage context effectively, an AI OS can run a nightly routine ('dreaming') that reviews daily memory files, compresses key information, and saves it into a long-term memory file. This process mimics human memory consolidation, preventing context loss over time.

A new OpenClaw feature called "dreaming" allows the AI agent to process information and consolidate memories overnight while inactive. This concept, borrowed from human neuroscience, aims to improve the agent's long-term learning and performance without requiring active user input, mimicking how humans process experiences during sleep.

A genuinely continual learner doesn't have separate training and testing phases. Instead, its life is a continuous process divided into two modes: an 'active' phase of interacting with new data and an 'offline' sleep phase for memory consolidation and self-improvement.

The key to continual learning is not just a longer context window, but a new architecture with a spectrum of memory types. "Nested learning" proposes a model with different layers that update at different frequencies—from transient working memory to persistent core knowledge—mimicking how humans learn without catastrophic forgetting.