We scan new podcasts and send you the top 5 insights daily.
Simple leaderboards tracking token usage lead to 'token maxing'—engineers burning tokens to look productive. A better approach is to use hack days and demos to reward and showcase high-impact output, which implicitly encourages effective AI use.
When companies give employees AI token budgets and track usage on dashboards, it incentivizes ROI-negative behavior. Employees feel compelled to spend their entire allocation to appear productive, a classic example of Goodhart's Law where the metric (usage) undermines the goal (productivity).
By ranking engineers on AI token consumption, Meta is experiencing Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." Employees reportedly build bots to needlessly burn tokens for status, demonstrating how gamifying a proxy metric can backfire and disconnect from actual business impact.
Companies like Meta are pushing a new practice called "token maxing," where developers are encouraged to spend heavily on AI coding assistant tokens. This is being gamified with leaderboards to accelerate output, but it raises questions about efficiency versus vanity metrics and whether it's a true indicator of productivity.
When companies measure AI adoption by counting tokens used, it creates a perverse incentive. Employees and their teams create agents to perform pointless tasks simply to boost their metrics, leading to fake productivity and problematic artifacts.
In the current 'capability exploration' phase, companies incentivize developers to use as many AI tokens as possible. This serves as a visible, albeit inefficient, signal of AI adoption to management, prioritizing quantity over quality.
A trend called "tokenmaxxing" is emerging in Silicon Valley, where companies like Meta use leaderboards to track employee AI token usage. This reflects a corporate bet that higher token consumption correlates with increased productivity, turning AI usage into a new, albeit gameable, performance metric for engineers.
Gamifying AI token consumption via internal leaderboards, as seen at Meta, creates perverse incentives. Employees may burn tokens to climb the ranks rather than to solve real business problems. This "tokenmaxxing" promotes conspicuous consumption of compute, a vanity metric that masks true productivity and ROI.
The trend of "token maxing" dashboards in companies like Meta leads to ROI-negative behavior. Employees engage in low-value tasks, like checking the weather, simply to climb a usage leaderboard, driven by a combination of Jevons Paradox and Goodhart's Law.
According to Mike Cannon-Brookes, advanced enterprises are not tracking AI success by counting tokens. Instead, they are asking harder questions about overall output, such as engineering productivity and quality. They understand that high token usage doesn't always correlate with high productivity, shifting focus from raw usage to tangible business outcomes.
At companies like Meta, a new practice called "token maxing" is being used to measure productivity, where engineers compete on leaderboards to consume the most AI tokens. Promoted by leaders from Nvidia and Meta, this metric is criticized for being easily gamed and not necessarily reflecting true productivity.