Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Kirby Montgomery says the audit reviews AI engineering and the product layer because many companies built useless chatbots or AI features nobody uses. For fine-tuned models, MCP servers, or other AI features, he asks who uses it, what outcome it drives, and what it costs per unit.

Related Insights

Because modern tooling and AI make coding features cheaper and faster, the primary business risk shifts from development cost to feature adoption. Flooding products with unvetted features increases downstream costs, as companies still incur the expense of driving user engagement.

Instead of asking "Can the model do this?", product leaders should ask four critical questions about the user experience, who builds the reliability layer, the business cost of its failure, and the cost to maintain it at scale.

Erickson says AI becomes a scaling resource only after you have demand and product-market fit; without that, you're optimizing nothing. He warns teams spend most AI time building things that don't need to exist. Focus AI on work that moves the needle, increases productivity, or improves conversion rather than just building cool stuff.

Before launch, product leaders must ask if their AI offering is a true product or just a feature. Slapping an AI label on a tool that automates a minor part of a larger workflow is a gimmick. It will fail unless it solves a core, high-friction problem for the customer in its entirety.

The excitement around AI often overshadows its practical business implications. Implementing LLMs involves significant compute costs that scale with usage. Product leaders must analyze the ROI of different models to ensure financial viability before committing to a solution.

In a market where customers eagerly pay for valuable AI tools, an inability to monetize new AI features is a major red flag. It indicates the product lacks sufficient value. A key test is whether AI can drive average revenue per user (ARPU) up by 50% or more; anything less is just a feature, not a transformation.

Unlike traditional SaaS where unit costs stay relatively flat, AI features incur per-interaction token expenses. Product and engineering teams must track costs down to workflows and users in the codebase to prevent uncontrolled margin erosion.

Founders can get lost building complex AI systems and automations. This can become a trap, a "procrastination machine," that feels productive but doesn't contribute to the primary goal of generating revenue. Always ask if the AI work is actually making the business money.

Unlike traditional SaaS, achieving product-market fit in AI doesn't guarantee a viable business. The high cost of goods sold (COGS) from model inference can exceed revenue, causing companies to lose more money as they scale. This forces a focus on economical model deployment from day one.

A cheap model that fails often becomes expensive due to retries, fallbacks, and human review. The true measure of economic efficiency is the cost to reliably complete a task, not the raw inference cost, which can be a misleading metric at scale.