Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

An effective AI agent isn't a single, all-knowing model. It's custom software that uses LLMs for inference only when necessary, relying on cheaper, deterministic code for most tasks. The goal is to maximize outcomes, not token usage, by blending AI with traditional software.

Related Insights

For core business automations, agentic AIs that "guess" are expensive and unreliable. A superior approach uses tools that convert natural language into deterministic, code-like workflows, which run consistently and use AI only when necessary.

Fully autonomous agents are not yet reliable for complex production use cases because accuracy collapses when chaining multiple probabilistic steps. Zapier's CEO recommends a hybrid "agentic workflow" approach: embed a single, decisive agent within an otherwise deterministic, structured workflow to ensure reliability while still leveraging LLM intelligence.

An LLM shouldn't do math internally any more than a human would. The most intelligent AI systems will be those that know when to call specialized, reliable tools—like a Python interpreter or a search API—instead of attempting to internalize every capability from first principles.

AI platforms using the same base model (e.g., Claude) can produce vastly different results. The key differentiator is the proprietary 'agent' layer built on top, which gives the model specific tools to interact with code (read, write, edit files). A superior agent leads to superior performance.

The true building block of an AI feature is the "agent"—a combination of the model, system prompts, tool descriptions, and feedback loops. Swapping an LLM is not a simple drop-in replacement; it breaks the agent's behavior and requires re-engineering the entire system around it.

The 'agents vs. applications' debate is a false dichotomy. Future applications will be sophisticated, orchestrated systems that embed agentic capabilities. They will feature multiple LLMs, deterministic logic, and robust permission models, representing an evolution of software, not a replacement of it.

Jerry Murdock predicts agents will use an orchestration layer to triage tasks, selecting the best LLM for each job—like expensive Claude for reasoning and cheap open-source models for simple tasks. This shifts value from the models themselves to the agent's intelligent orchestration capabilities.

A hybrid approach to AI agent architecture is emerging. Use the most powerful, expensive cloud models like Claude for high-level reasoning and planning (the "CEO"). Then, delegate repetitive, high-volume execution tasks to cheaper, locally-run models (the "line workers").

Top-tier language models are becoming commoditized in their excellence. The real differentiator in agent performance is now the 'harness'—the specific context, tools, and skills you provide. A minimalist, well-crafted harness on a good model will outperform a bloated setup on a great one.

With most large models crossing a "good enough" intelligence threshold, the competitive advantage for AI agents is shifting. It's no longer about using the single smartest model, but about building a system that can intelligently route tasks to a variety of models to optimize for price, performance, and specific use cases.

AI Agents Are Not 'God in a Box' but Custom Software Using LLMs Sparingly | RiffOn