Before considering expensive model fine-tuning, implement Retrieval-Augmented Generation (RAG). RAG dynamically retrieves information from a knowledge base to augment the prompt, solving most domain-specific problems efficiently. The recommended hierarchy is: Prompt Optimization -> Context Engineering -> RAG -> Fine-tuning.

Retrieval-Augmented Generation (RAG) is just one component of agent memory. A robust system must also handle dynamic operations like updating information, consolidating knowledge, resolving conflicts, and strategically forgetting obsolete data.

While vector search is a common approach for RAG, Anthropic found it difficult to maintain and a security risk for enterprise codebases. They switched to "agentic search," where the AI model actively uses tools like grep or find to locate code, achieving similar accuracy with a cleaner deployment.

Vector similarity does not equal relevance. A lightweight re-ranker model, placed between retrieval and generation, rescores search results for usefulness. This significantly improves performance on ambiguous queries and is one of the highest-impact additions to a RAG pipeline.

Teams often agonize over which vector database to use for their Retrieval-Augmented Generation (RAG) system. However, the most significant performance gains come from superior data preparation, such as optimizing chunking strategies, adding contextual metadata, and rewriting documents into a Q&A format.

Before implementing complex solutions like agentic loops, a highly effective first step for improving RAG performance is simply increasing the number of retrieved chunks (k). This simple change fixed nearly half of the initial flagged query failures by preventing relevant notes with similar chunks from consuming the entire context budget.

Classic RAG involves a single data retrieval step. Its evolution, "agentic retrieval," allows an AI to perform a series of conditional fetches from different sources (APIs, databases). This enables the handling of complex queries where each step informs the next, mimicking a research process.

Startups that built complex, bespoke systems like RAG for early models now face a dilemma. These systems, once an advantage, can become technical debt as platform-native capabilities (like file search or simple `grep`) surpass them, making it difficult to stay on the frontier.

The nature of Retrieval-Augmented Generation (RAG) is evolving. Instead of a single search to populate an initial context window, AI agents are now performing numerous concurrent queries in a single turn. This allows them to explore diverse information paths simultaneously, driving new database requirements.