A powerful cost-saving strategy is to use AI as a one-time tool to generate complex, deterministic code for a recurring problem. This avoids the high, cumulative cost of running the same reasoning task through a pay-per-use LLM, shifting the expense from operational credits to a one-time development effort.

While most focus on building more power infrastructure to meet AI's energy needs, the truly disruptive innovation may come from creating chips and models that are massively more energy-efficient. This contrarian view suggests the real investment opportunity might be in demand-side technology, not just supply-side energy production.

Designing a chip is not a monolithic problem that a single AI model like an LLM can solve. It requires a hybrid approach. While LLMs excel at language and code-related stages, other components like physical layout are large-scale optimization problems best solved by specialized graph-based reinforcement learning agents.

Model architecture decisions directly impact inference performance. AI company Zyphra pre-selects target hardware and then chooses model parameters—such as a hidden dimension with many powers of two—to align with how GPUs split up workloads, maximizing efficiency from day one.

Running premium AI models constantly is prohibitively expensive. A cost-effective strategy is to use a cheaper model as a "manager" to understand a high-level goal, break it down, and then delegate the execution of sub-tasks to multiple, short-lived "child" sessions running more powerful models.

Adding more FLOPS to current AI chips is useless due to thermal throttling. Etched realized the solution is lowering voltage, which quadratically reduces power consumption. Inspired by bitcoin miners, they created a new power delivery system enabling chips to run at under half the voltage of GPUs.

Current AI models become exponentially more expensive as input size grows (quadratic scaling). New "subquadratic" architectures, however, scale linearly by pre-selecting relevant data. This change could slash compute costs by orders of magnitude, making massive context windows economically viable.

Chinese AI models like Kimi achieve dramatic cost reductions through specific architectural choices, not just scale. Using a "mixture of experts" design, they only utilize a fraction of their total parameters for any given task, making them far more efficient to run than the "dense" models common in the West.

To manage costs, the optimal architecture isn't running everything on the most powerful model. Instead, a smart orchestrator agent should break down complex problems and dispatch simpler sub-tasks to smaller, cheaper models, optimizing for both cost and performance.