Adopt a "start simple" approach for AI development. Master prompting first. If that fails, use Retrieval Augmented Generation (RAG). Fine-tuning should be the last resort due to its complexity in deployment, serving, and keeping up with rapidly evolving base models.

Embedding-based RAG for code search is falling out of favor because its arbitrary chunking often fails to capture full semantic context. Simpler, more direct approaches like agent-based search using tools like `grep` are proving more reliable and scalable for retrieving relevant code without the maintenance overhead of embeddings.

Overly structured, workflow-based systems that work with today's models will become bottlenecks tomorrow. Engineers must be prepared to shed abstractions and rebuild simpler, more general systems to capture the gains from exponentially improving models.

The opportunity cost of building custom internal AI can be massive. By the time a multi-million dollar project is complete, off-the-shelf tools like ChatGPT are often far more capable, dynamic, and cost-effective, rendering the custom solution outdated on arrival.

Before considering expensive model fine-tuning, implement Retrieval-Augmented Generation (RAG). RAG dynamically retrieves information from a knowledge base to augment the prompt, solving most domain-specific problems efficiently. The recommended hierarchy is: Prompt Optimization -> Context Engineering -> RAG -> Fine-tuning.

While vector search is a common approach for RAG, Anthropic found it difficult to maintain and a security risk for enterprise codebases. They switched to "agentic search," where the AI model actively uses tools like grep or find to locate code, achieving similar accuracy with a cleaner deployment.

The "bitter lesson" of AI applies to product development: complex scaffolding built around model limitations (like early vector stores or agent frameworks) will inevitably become obsolete as the models themselves get smarter and absorb those functions. Don't over-engineer solutions that a future model will solve natively.

Anthropic's "Managed Agents" is built on the premise that any specific "harness" is temporary, as its assumptions become outdated with model improvements. They are creating a "meta-harness"—an underlying infrastructure designed to outlast any single implementation, making individual harnesses easily swappable and disposable.

The nature of Retrieval-Augmented Generation (RAG) is evolving. Instead of a single search to populate an initial context window, AI agents are now performing numerous concurrent queries in a single turn. This allows them to explore diverse information paths simultaneously, driving new database requirements.