Many AI coding agents are unprofitable because their business model is broken. They charge a fixed subscription fee but pay variable, per-token costs for model inference. This means their most engaged power users, who should be their best customers, are actually their biggest cost centers, leading to negative gross margins.

The excitement around AI often overshadows its practical business implications. Implementing LLMs involves significant compute costs that scale with usage. Product leaders must analyze the ROI of different models to ensure financial viability before committing to a solution.

Historically, a developer's primary cost was salary. Now, the constant use of powerful AI coding assistants creates a new, variable infrastructure expense for LLM tokens. This changes the economic model of software development, with costs per engineer potentially rising by dollars per hour.

Relying solely on premium models like Claude Opus can lead to unsustainable API costs ($1M/year projected). The solution is a hybrid approach: use powerful cloud models for complex tasks and cheaper, locally-hosted open-source models for routine operations.

While the cost to achieve a fixed capability level (e.g., GPT-4 at launch) has dropped over 100x, overall enterprise spending is increasing. This paradox is explained by powerful multipliers: demand for frontier models, longer reasoning chains, and multi-step agentic workflows that consume exponentially more tokens.

Heavy use of AI agents and API calls is generating significant costs, with some agents costing $100,000 annually. This creates a new financial reality where companies must budget for 'tokens' per employee, potentially making the AI's cost more than the human's salary.

While the cost for GPT-4 level intelligence has dropped over 100x, total enterprise AI spend is rising. This is driven by multipliers: using larger frontier models for harder tasks, reasoning-heavy workflows that consume more tokens, and complex, multi-turn agentic systems.

The primary driver for running AI models on local hardware isn't cost savings or privacy, but maintaining control over your proprietary data and models. This avoids vendor lock-in and prevents a third-party company from owning your organization's 'brain'.

The success of personal AI assistants signals a massive shift in compute usage. While training models is resource-intensive, the next 10x in demand will come from widespread, continuous inference as millions of users run these agents. This effectively means consumers are buying fractions of datacenter GPUs like the GB200.