While AI training is data-center-intensive, Cisco's CEO sees the move to AI inference as a massive growth opportunity. Inference will happen at distributed edge locations to be close to users, requiring robust, high-performance networks to connect everything, which plays directly into the company's core strengths.

While often discussed for privacy, running models on-device eliminates API latency and costs. This allows for near-instant, high-volume processing for free, a key advantage over cloud-based AI services.

Relying on third-party APIs for AI is becoming unsustainable due to high token costs and the inherent security risk of uploading sensitive data. This will force a market shift toward powerful local hardware for running private, cost-effective models.

The inherent limitations of edge environments, such as privacy concerns and the need for low-latency responses, are not just technical hurdles. They represent the core value propositions driving the adoption of edge AI, as it solves these problems directly where data is generated.

Akamai leverages its historic strength in edge networking for its compute offering. By allowing customers to build and deliver applications at the edge, closer to users, they can significantly reduce expensive egress fees typically charged by traditional hyperscale cloud providers. This cost-saving angle is a key competitive differentiator.

The recent economic push for AI to demonstrate a clear return on investment is not new to the edge AI space. Edge applications have always been driven by strict cost and productivity constraints, fostering a culture of rational, value-focused development that the broader AI world is now adopting.

While AI training requires massive, centralized data centers, the growth of inference workloads is creating a need for a new architecture. This involves smaller (e.g., 5 megawatt), decentralized clusters located closer to users to reduce latency. This shift impacts everything from data center design to the software required to manage these distributed fleets.

Rivian is adding powerful AI hardware to its cars for edge computing. The business case isn't just better performance; over the long run, processing AI requests locally reduces reliance on cloud servers, saving significant future costs on data connectivity and cloud-based inference.

The primary driver for running AI models on local hardware isn't cost savings or privacy, but maintaining control over your proprietary data and models. This avoids vendor lock-in and prevents a third-party company from owning your organization's 'brain'.