Inspired by human dreaming as a memory reconsolidation process, Anthropic has its AI agents use downtime to "dream." During this background process, the agent reviews its memories, identifies and prunes contradictions, and cleans up the information to improve the coherence and utility of its long-term memory.

Effective agent memory is not merely a storage layer. It's an encapsulated system for learning and adaptation that integrates embedding models, re-rankers, databases, and LLMs, all working in concert to hold, move, and store data.

Retrieval-Augmented Generation (RAG) is just one component of agent memory. A robust system must also handle dynamic operations like updating information, consolidating knowledge, resolving conflicts, and strategically forgetting obsolete data.

Instead of treating memory as a component, adopt a "memory-first" approach when designing agent systems. This paradigm shift involves architecting the entire system around the core principles of how information is stored, recalled, and forgotten.

To solve the problem of an AI agent creating low-quality memories and skills ("slop") over time, Hermes Agent runs a sub-system called "Hermes Curator." This internal agent automatically and continuously cleans, refines, and improves the main agent's learned skills and memories.

Google's Titans architecture for LLMs mimics human memory by applying Claude Shannon's information theory. It scans vast data streams and identifies "surprise"—statistically unexpected or rare information relative to its training data. This novel data is then prioritized for long-term memory, preventing clutter from irrelevant information.

To manage huge context sizes, Lindy uses "recursive context buckets" organized in a self-balancing tree. This data structure allows an AI agent to access information from a context of billions of tokens with just two LLM calls, effectively solving the context window limitation for complex tasks.

Seemingly complex features like long-term memory and skill creation are fundamentally clever systems for managing an AI's limited context window. The "harness" efficiently loads and unloads relevant information (memories, skills) at the precise moment it's needed, rather than keeping it all in context constantly.

Instead of curating a personal knowledge base, feed raw information (articles, posts, data) to AI agents. Task them with organizing it, identifying patterns, and forming rules. This creates a system where the agents' effectiveness grows autonomously with new data.