An agent's procedural memory (its skills) is analogous to a human's Standard Operating Procedures (SOPs). Storing these "SOPs"—such as in markdown files—inside a database allows them to be selectively retrieved, enabling the agent to scale its capabilities.

Effective agent memory is not merely a storage layer. It's an encapsulated system for learning and adaptation that integrates embedding models, re-rankers, databases, and LLMs, all working in concert to hold, move, and store data.

AI agents are powerful but amnestic. They need a "heartbeat" checklist—a set of standing instructions—to re-orient themselves on their identity, goals, and tasks every time they activate, just like the protagonist of the film "Memento."

Retrieval-Augmented Generation (RAG) is just one component of agent memory. A robust system must also handle dynamic operations like updating information, consolidating knowledge, resolving conflicts, and strategically forgetting obsolete data.

Instead of treating memory as a component, adopt a "memory-first" approach when designing agent systems. This paradigm shift involves architecting the entire system around the core principles of how information is stored, recalled, and forgotten.

An AI model alone is like a brain without a body. To become a useful agent, it needs a "harness" or "scaffolding" consisting of four key components: domain-specific knowledge, memory of past interactions, tools to take actions, and guardrails for safety.

Instead of just expanding context windows, the next architectural shift is toward models that learn to manage their own context. Inspired by Recursive Language Models (RLMs), these agents will actively retrieve, transform, and store information in a persistent state, enabling more effective long-horizon reasoning.

The "memory" feature in today's LLMs is a convenience that saves users from re-pasting context. It is far from human memory, which abstracts concepts and builds pattern recognition. The true unlock will be when AI develops intuitive judgment from past "experiences" and data, a much longer-term challenge.

The key to continual learning is not just a longer context window, but a new architecture with a spectrum of memory types. "Nested learning" proposes a model with different layers that update at different frequencies—from transient working memory to persistent core knowledge—mimicking how humans learn without catastrophic forgetting.