Open source AI models can't improve in the same decentralized way as software like Linux. While the community can fine-tune and optimize, the primary driver of capability—massive-scale pre-training—requires centralized compute resources that are inherently better suited to commercial funding models.

Unlike general-purpose cloud resources, AI training infrastructure with specialized networking (e.g., InfiniBand) and storage cannot be added fungibly. It requires significant pre-planning and deep integration, breaking the standard cloud deployment model of simply adding more commoditized compute or storage as needed.

While AI inference can be decentralized, training the most powerful models demands extreme centralization of compute. The necessity for high-bandwidth, low-latency communication between GPUs means the best models are trained by concentrating hardware in the smallest possible physical space, a direct contradiction to decentralized ideals.

The primary bottleneck for Project Maven wasn't algorithms but outdated digital infrastructure. Data packets crisscrossing the Atlantic multiple times and physical hardware encryptors creating bottlenecks revealed that cutting-edge AI is useless without a modernized, high-throughput network to support it.

Mixture-of-Experts (MoE) models require an "all-to-all" communication pattern. This is efficient within a single GPU rack's high-speed interconnect but becomes a major bottleneck between racks, where communication is ~8x slower. This effectively limits an MoE layer's maximum size to what a single rack can support.

When splitting jobs across thousands of GPUs, inconsistent communication times (jitter) create bottlenecks, forcing the use of fewer GPUs. A network with predictable, uniform latency enables far greater parallelization and overall cluster efficiency, making it more important than raw 'hero number' bandwidth.

While the idea of distributed compute pools is appealing, it's not feasible for AI training due to high latency demands; GPUs must be physically co-located. However, AI inference is less sensitive to this lag, making a distributed network of compute (like home GPUs) a much more viable and exciting model.

Pre-training requires constant, high-bandwidth weight synchronization, making it difficult across data centers. Newer Reinforcement Learning (RL) methods mostly do local forward passes to generate data, only sending back small amounts of verified data, making distributed training more practical.

As single data centers hit power limits, AI training clusters are expanding across locations hundreds of kilometers apart. This "scale across" model creates a new engineering challenge: preventing packet loss, which can ruin expensive training runs. The solution lies in silicon-level innovations like deep buffering to maintain coherence over long distances.