The key measure of leverage for AI-powered developers is no longer GPU utilization (FLOPs) but the volume of tokens processed by agents. Karpathy feels nervous when his token subscriptions are underutilized, indicating he's the bottleneck, not the system.

Goal-based loops run until an outcome is validated. If the success criteria are poorly defined, the agent will continuously burn tokens in a potentially fruitless effort. This makes precise prompt engineering and evaluation criteria critical for cost control.

When evaluating AI agents, the total cost of task completion is what matters. A model with a higher per-token cost can be more economical if it resolves a user's query in fewer turns than a cheaper, less capable model. This makes "number of turns" a primary efficiency metric.

Progress in complex, long-running agentic tasks is better measured by tokens consumed rather than raw time. Improving token efficiency, as seen from GPT-5 to 5.1, directly enables more tool calls and actions within a feasible operational budget, unlocking greater capabilities.

Move beyond manual agent improvement by creating an automated loop. In this process, an agent runs, its performance is evaluated, failures are identified, and another process suggests and implements code fixes. This creates a foundation for self-improving systems.

High token consumption is framed as a key metric for AI leverage, not a cost. This goal forces teams to find ways to delegate more complex, long-running, and parallel tasks to AI agents, thus maximizing the intelligence and autonomous work extracted from the models.

Don't pass the full, token-heavy output of every tool call back into an agent's message history. Instead, save the raw data to an external system (like a file system or agent state) and only provide the agent with a summary or pointer.

In complex, multi-step tasks, overall cost is determined by tokens per turn and the total number of turns. A more intelligent, expensive model can be cheaper overall if it solves a problem in two turns, while a cheaper model might take ten turns, accumulating higher total costs. Future benchmarks must measure this turn efficiency.

The simple "tool calling in a loop" model for agents is deceptive. Without managing context, token-heavy tool calls quickly accumulate, leading to high costs ($1-2 per run), hitting context limits, and performance degradation known as "context rot."