Technical metrics like "accuracy" are often the wrong measure for ML projects and can mismanage expectations. To achieve success, projects must be evaluated using business KPIs like profit, savings, or ROI. This aligns data science with business goals and reveals the true value of imperfect predictions.

An AI pilot study defined success as "marked and sustained" profit impact within six months. This impossibly high bar automatically classified projects that broke even, were on track for future profit, or provided non-financial benefits as "failures," thus obscuring the real, incremental value of new technology deployments.

Most AI ROI models are optimistic projections, not true business cases. They fail because their financial assumptions about user adoption, data availability, and decision speed don't account for the fragmented governance and misaligned incentives that are constraining the organization. The model assumes a reality that doesn't exist.

Before concluding a company can sustain extraordinary growth, consult historical data ('base rates') on how many similar companies succeeded in the past. This 'outside view,' a concept from investor Michael Mauboussin, provides a crucial reality check against overly optimistic forecasts.

A CFO doesn't care that AI can summarize literature faster. They care that faster synthesis shortens publication times, accelerates HCP uptake, and impacts sales by a quantifiable amount. A credible financial case must map the entire chain of causality from an AI capability to a specific, revenue-driving business decision.

To avoid emotional spending that kills runway, analyze every major decision through three financial scenarios. A 'bear' case (e.g., revenue drops 10%), 'base' case (plan holds), and 'bull' case (revenue grows 10%). This sobering framework forces you to quantify risk and compare alternatives objectively before committing capital.

Teams often build financial models to confirm their enthusiasm for a particular AI initiative. However, the model's greatest value comes from rigorously challenging these assumptions. Often, the most hyped projects are revealed to have a fraction of the financial impact of less visible but more strategic alternatives.

Aim for "good enough" financial estimates to differentiate multi-million dollar opportunities from thousand-dollar ones. This high-level sorting is more valuable and efficient than creating detailed, yet still speculative, forecasts for every idea.

When pitching a risky hypothesis, anticipate skepticism by pressure-testing your own assumptions beforehand. Presenting financial models for multiple scenarios (e.g., a 10% vs. 20% win rate increase) demonstrates rigor and can win over skeptics, as even the worst-case outcome can still be a net positive for the business.