We scan new podcasts and send you the top 5 insights daily.
While it's easy to measure increased output from AI, like completing more story points, product leaders are failing to connect these efficiency gains to actual business ROI or customer value. This creates a significant blind spot when justifying AI investments.
Don't just report on leading indicators like faster cycle times. You must explicitly connect them to forecasted lagging outcomes. Present a clear narrative showing how today's efficiency gain will translate into future revenue or cost savings, providing a range of potential impacts.
The success of AI in marketing should not be measured by the quantity of content or ideas generated, which can create chaos. Instead, leaders must track its impact on core business metrics like revenue growth and operational efficiency. The goal is enabling a 10-person team to operate with the impact of a 100-person team.
Technical metrics like "accuracy" are often the wrong measure for ML projects and can mismanage expectations. To achieve success, projects must be evaluated using business KPIs like profit, savings, or ROI. This aligns data science with business goals and reveals the true value of imperfect predictions.
Beyond saving developer hours, the true value of AI-driven efficiency lies in reducing rework. This frees up capacity for new revenue-generating projects. Frame the value not just as time saved, but as the business value of features you can now build instead (cost of delay).
Demanding a direct, line-item ROI for foundational AI initiatives is like asking for the ROI on Wi-Fi—it's the wrong question. Instead of getting bogged down in impossible calculations, leaders should focus on measuring the business outcomes enabled by the technology, such as innovation speed or new product creation. Obsess on outcomes, not direct financial return.
According to Mike Cannon-Brookes, advanced enterprises are not tracking AI success by counting tokens. Instead, they are asking harder questions about overall output, such as engineering productivity and quality. They understand that high token usage doesn't always correlate with high productivity, shifting focus from raw usage to tangible business outcomes.
Businesses are unlikely to use powerful AI simply to shave a few percentage points off their software spend. The real, high-impact ROI comes from applying AI to improve core business operations, making the actual business more effective and efficient.
Companies struggle to measure AI's return on investment because its value often materializes as individual productivity gains for employees. These personal efficiencies, like finishing work earlier, don't show up on corporate dashboards, creating a mismatch between perceived value and actual impact.
Leaders often expect AI to produce a shiny, marketable feature. When AI’s value is 'invisible'—baked into workflows to improve efficiency—translate those gains into concrete financial outcomes like cost savings or accelerated revenue, rather than focusing on the process improvements themselves.
Vanity metrics like "AI lines of code" are misleading. Coinbase measures AI success by its impact on the end-to-end development cycle: the total time from a ticket's creation to the change landing with a user. This metric holistically captures gains and focuses the team on true velocity.