Instead of focusing on headcount reduction, Goldman's CIO measures the success of developer AI tools by their ability to consistently help projects finish ahead of schedule. This provides a tangible metric for increased output and organizational capacity.

Intercom's CTO set a goal to 2x R&D throughput, using pull requests as a simple, albeit crude, metric. In a high-trust environment, this focused the team on adopting AI tools to increase output, leading to measurable success.

If your team cannot articulate the specific business outcome of their AI usage in a single sentence, you don't have an AI strategy. You simply have 'token maxing'—usage for the sake of usage. This framework forces a direct link between AI spend and business results.

Demanding a direct, line-item ROI for foundational AI initiatives is like asking for the ROI on Wi-Fi—it's the wrong question. Instead of getting bogged down in impossible calculations, leaders should focus on measuring the business outcomes enabled by the technology, such as innovation speed or new product creation. Obsess on outcomes, not direct financial return.

Ramp's CPO argues companies shouldn't excessively worry about AI token costs. If an AI agent can deliver 10x the output of a human, it's logical and profitable to pay the agent (via tokens) more than the human's salary. This reframes ROI from a cost center to a massive productivity investment.

According to Mike Cannon-Brookes, advanced enterprises are not tracking AI success by counting tokens. Instead, they are asking harder questions about overall output, such as engineering productivity and quality. They understand that high token usage doesn't always correlate with high productivity, shifting focus from raw usage to tangible business outcomes.

While objective studies on AI coding assistants are mixed, their enterprise ROI is easily justified. Executives approve the investment because their most valuable employees—engineers—report significant productivity gains, making the business case simple regardless of hard data.

Don't rely on traditional project milestones to gauge AI progress. Instead, measure success through granular unit economics and operational metrics. Metrics like 'cost per release' or 'cycle time per feature' provide immediate feedback on whether your strategic hypothesis is valid, enabling rapid iteration.

Vanity metrics like "AI lines of code" are misleading. Coinbase measures AI success by its impact on the end-to-end development cycle: the total time from a ticket's creation to the change landing with a user. This metric holistically captures gains and focuses the team on true velocity.