/
© 2026 RiffOn. All rights reserved.

Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

  1. Behind the Craft
  2. 5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway
5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft · Aug 9, 2026

Linear's team shares their playbook for building production AI agents: start simple, launch internally, and build native agents to embed opinions.

AI Agents Are Simply LLMs in a Loop That Build Their Own Context

At a technical level, an AI agent is a large language model that repeatedly calls itself. It uses provided tools to gather information and build its own context until it has enough data to achieve a predefined goal, making complex tasks autonomous.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Linear's AI Agent Uses a "Skills" Architecture to Avoid API Hallucinations

Directly giving an agent a full API schema causes hallucinations. Linear solved this by developing a "skills" architecture where the agent dynamically loads specific, pre-defined toolsets and instructions (e.g., "create an issue") based on the user's task.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Effective AI Agents Need Minimal Instructions and Tools to Load Their Own Context

Counterintuitively, giving an AI agent too much upfront context or overly specific instructions degrades its performance. The best practice is to provide minimal guidance and instead equip the agent with tools to dynamically fetch the necessary information itself.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Successful AI Agents Integrate into Workflow "On-Ramps" Like Slack, Not Just Chatbots

A chatbot is a necessary interface for multi-turn interactions but shouldn't be the primary entry point. The most effective domain-specific agents are accessible from natural "on-ramps" within a user's existing workflow, such as a Slack conversation or a meeting summary.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Linear Routes Common AI Agent Tasks to Specialized Small Models for Efficiency

Instead of using one large model for all tasks, Linear employs a small router model. For high-frequency use cases like creating issues, it routes the request to a smaller, highly-optimized model and prompt, saving costs while improving performance and reliability.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

AI Agent Evals Should Enforce Critical Consistency Without Stifling Natural Variance

When evaluating AI agents, it's crucial to distinguish between areas that require strict consistency (e.g., setting a ticket status correctly) and those that benefit from natural variance. Over-evaluating for consistency in creative responses can generate false negatives and limit the agent's utility.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Linear Discovered Emergent AI Use Cases by Stealth-Launching Within Its Existing Slack Bot

To understand how users would naturally interact with their agent, Linear quietly replaced its procedural Slack bot with the new AI. This revealed unexpected behaviors, like users simply typing "@linear do the right thing," providing invaluable real-world usage data.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

The Biggest Problem in AI Is Underutilization, Not Lack of Model Intelligence

The primary bottleneck in applied AI is not model capability but human integration. There is a massive "capacity overhang" where models are far more powerful than how we currently apply them in daily workflows. The focus should be on better application, not just better models.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Prototype AI Agents with the Largest Model Before Optimizing Down with Evals

When building a new agent feature, the best practice is to use the most powerful model available to explore the full capability space. Only after establishing clear success criteria and evaluation frameworks should you optimize by using smaller, more cost-effective models.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago

Native AI Agents Allow Companies to Codify Their Opinionated Workflows

A generic API lets users build their own processes, which can be inconsistent. By building a native agent, a company like Linear can embed its specific philosophy and best practices (e.g., "the Linear method") into the agent's core behavior, ensuring quality.

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway thumbnail

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Behind the Craft·2 months ago