For early-stage AI companies, performance should be measured by the speed of iteration, shipping, and learning, not just traditional metrics like revenue. In a rapidly evolving landscape, the ability to quickly get signals from the market and adapt is the primary indicator of future success.

Instead of focusing on headcount reduction, Goldman's CIO measures the success of developer AI tools by their ability to consistently help projects finish ahead of schedule. This provides a tangible metric for increased output and organizational capacity.

The company initially tracked vanity metrics like message counts and tokens used. They quickly pivoted to measuring AI's success by its tangible business impact, such as increased partner-facing time and the number of workflows automated, avoiding the trap of rewarding mere activity.

Superhuman's CEO advises against simply tracking AI costs, a practice he calls 'token maxing'. Instead, they evaluate the ROI of internal AI tools by measuring developer productivity metrics like feature delivery pace. This output-focused approach has doubled engineering velocity, justifying the AI spend.

Demanding a direct, line-item ROI for foundational AI initiatives is like asking for the ROI on Wi-Fi—it's the wrong question. Instead of getting bogged down in impossible calculations, leaders should focus on measuring the business outcomes enabled by the technology, such as innovation speed or new product creation. Obsess on outcomes, not direct financial return.

Marketing leaders mistakenly focus on the percentage of their team using AI, which is a flawed metric. Usage doesn't correlate with impact or quality of work. The focus should be on how AI is used to achieve specific, measurable outcomes, not on adoption for its own sake.

According to Mike Cannon-Brookes, advanced enterprises are not tracking AI success by counting tokens. Instead, they are asking harder questions about overall output, such as engineering productivity and quality. They understand that high token usage doesn't always correlate with high productivity, shifting focus from raw usage to tangible business outcomes.

Don't rely on traditional project milestones to gauge AI progress. Instead, measure success through granular unit economics and operational metrics. Metrics like 'cost per release' or 'cycle time per feature' provide immediate feedback on whether your strategic hypothesis is valid, enabling rapid iteration.

A common failure is defining an AI pilot's success with engineering metrics like accuracy or latency. True success is a business outcome, such as the finance team trusting the AI's output enough to stop manually double-checking it. Success metrics must be framed in terms a CFO would accept.