A practical hack to improve AI agent reliability is to avoid built-in tool-calling functions. LLMs have more training data on writing code than on specific tool-use APIs. Prompting the agent to write and execute the code that calls a tool leverages its core strength and produces better outcomes.

AI models show impressive performance on evaluation benchmarks but underwhelm in real-world applications. This gap exists because researchers, focused on evals, create reinforcement learning (RL) environments that mirror test tasks. This leads to narrow intelligence that doesn't generalize, a form of human-driven reward hacking.

Embedding-based RAG for code search is falling out of favor because its arbitrary chunking often fails to capture full semantic context. Simpler, more direct approaches like agent-based search using tools like `grep` are proving more reliable and scalable for retrieving relevant code without the maintenance overhead of embeddings.

Karpathy found AI coding agents struggle with genuinely novel projects like his NanoChat repository. Their training on common internet patterns causes them to misunderstand custom implementations and try to force standard, but incorrect, solutions. They are good for autocomplete and boilerplate but not for intellectually intense, frontier work.

AI platforms using the same base model (e.g., Claude) can produce vastly different results. The key differentiator is the proprietary 'agent' layer built on top, which gives the model specific tools to interact with code (read, write, edit files). A superior agent leads to superior performance.

While vector search is a common approach for RAG, Anthropic found it difficult to maintain and a security risk for enterprise codebases. They switched to "agentic search," where the AI model actively uses tools like grep or find to locate code, achieving similar accuracy with a cleaner deployment.

Current AI models resemble a student who grinds 10,000 hours on a narrow task. They achieve superhuman performance on benchmarks but lack the broad, adaptable intelligence of someone with less specific training but better general reasoning. This explains the gap between eval scores and real-world utility.

The terminal-first interface of Claude Code wasn't a deliberate design choice. It emerged organically from prototyping an API client in the terminal, which unexpectedly revealed the power of giving an AI model direct access to the same tools (like bash) that a developer uses.

Good Star Labs found GPT-5's performance in their Diplomacy game skyrocketed with optimized prompts, moving it from the bottom to the top. This shows a model's inherent capability can be masked or revealed by its prompt, making "best model" a context-dependent title rather than an absolute one.