To quantify the real-world impact of its AI tools, Block tracks a simple but powerful metric: "manual hours saved." This KPI combines qualitative and quantitative signals to provide a clear measure of ROI, with a target to save 25% of manual hours across the company.

Instead of measuring how many questions a chatbot answered, the bank tracked the reduction in support tickets for the corresponding human team, which dropped 60%. This focus on tangible business outcomes provides a much clearer picture of AI's real value than simple activity metrics.

Unlike traditional software that optimizes for time-in-app, the most successful AI products will be measured by their ability to save users time. The new benchmark for value will be how much cognitive load or manual work is automated "behind the scenes," fundamentally changing the definition of a successful product.

Instead of vanity usage metrics, Wiz focuses on a core customer outcome: helping customers resolve all critical risks. They gamified this by creating the 'Zero Criticals Club.' This metric proves the product is driving real organizational change, a key indicator of value and stickiness that is hard to replace.

In a digital-first world, measuring success by the number of assets produced is meaningless. Leaders must shift to outcome-based metrics like speed from idea to launch, brand effectiveness, and direct impact on engagement and conversion to gauge true performance.

Focus on what customers value (e.g., delivery speed, order accuracy) rather than internal business metrics like ARR or user growth. This approach naturally leads to a better product roadmap and a more defensible business by solving real user problems.

In government, digital services are often viewed as IT projects delivered by contractors. A CPO's primary challenge is instilling a culture of product thinking: focusing on customer value, business outcomes, user research, and KPIs, often starting from a point of zero.

Product teams focus on technical metrics like scalability, but customer-facing teams see success differently: it's when a client says they "couldn't run their business" without the product. The goal is to merge these two definitions by translating technical achievements into tangible customer outcomes.

Vanity metrics like "AI lines of code" are misleading. Coinbase measures AI success by its impact on the end-to-end development cycle: the total time from a ticket's creation to the change landing with a user. This metric holistically captures gains and focuses the team on true velocity.