Early AI adoption metrics focused on usage (e.g., tokens consumed), leading to wasteful “token maxing.” Successful teams quickly pivoted to measuring real business impact, such as the overall speed of the software delivery lifecycle, to gauge AI effectiveness and drive true ROI.

Enterprises that track and reward high AI token usage risk incentivizing the wrong behavior. This is a modern "Cobra Effect," where employees generate unnecessary output to hit metrics, much like people who bred cobras to collect a bounty. The focus must be on utility, not volume.

The company initially tracked vanity metrics like message counts and tokens used. They quickly pivoted to measuring AI's success by its tangible business impact, such as increased partner-facing time and the number of workflows automated, avoiding the trap of rewarding mere activity.

While many teams track token usage as a primary AI ROI metric, Anthropic views it differently. High token usage doesn't correlate with high value, as it's easy to waste tokens. Instead, a token count of zero is the most important signal, as it clearly indicates someone is not using the AI at all.

The current AI hype cycle can create misleading top-of-funnel metrics. The only companies that will survive are those demonstrating strong, above-benchmark user and revenue retention. It has become the ultimate litmus test for whether a product provides real, lasting value beyond the initial curiosity.

Many AI products suffer from outrageous churn (20-40% monthly) because they promise full automation but deliver mediocre, "80% there" results that still require human oversight. The key to a successful AI agent business is building a brand reputation for being exceptionally good and reliable, not just average.

To evaluate AI's role in building relationships, marketers must look beyond transactional KPIs. Leading indicators of success include sustained engagement, customers volunteering more information, and recommending the experience to others. These metrics quantify brand trust and empathy—proving the brand is earning belief, not just attention.

According to Mike Cannon-Brookes, advanced enterprises are not tracking AI success by counting tokens. Instead, they are asking harder questions about overall output, such as engineering productivity and quality. They understand that high token usage doesn't always correlate with high productivity, shifting focus from raw usage to tangible business outcomes.

The initial approach to AI adoption was often "token maxing"—using as many tokens as possible under the assumption that more usage equals more value. A more sophisticated and sustainable strategy is "output maxing," which focuses on achieving the desired result while actively minimizing token consumption and cost.