The AI 2040 plan suggests new AI R&D data centers have nation-state-level physical security, including Faraday cages and air-gapped communications. To prevent model theft, they propose capping external connections at 1 MB/s, making it take years to exfiltrate large model weights.
Contrary to the 'stop AI' narrative, the AI 2040 proposal advocates for continued use of current models and capabilities research. The explicit goal is not to halt progress but to slow the pace of development to ensure safety, targeting superintelligence for 2040 instead of a rushed 2028 timeline.
A novel safety proposal involves intentionally training models to be larger than computationally optimal. By increasing model weights to 100 terabytes instead of a more efficient one terabyte, the physical difficulty and time required to move or steal the model increases dramatically, creating a practical security barrier.
The argument that AI doomers should short the market if they truly believe in existential risk is flawed. As pointed out in the discussion, such bets are nonsensical because in a world-ending scenario, there would be no one left to collect the winnings, making it an impractical measure of belief.
While aimed at safety, proposed regulations could inadvertently create massive barriers to entry. The high cost and complexity of compliance would favor a few large, established AI labs, effectively giving them regulatory capture and stifling competition from smaller players and startups who can't make the cut.
The successful software simulation of a fruit fly's neural network has led to experiments like trapping it in a virtual box. This raises nascent ethical questions about the moral weight of actions taken against simulated beings, foreshadowing complex debates about the rights of more advanced synthetic intelligences.
The AI 2040 plan's call for major countries to declare AI compute inventories and jointly escort model weights mirrors the complex, often fraught, world of nuclear non-proliferation. This highlights a critical vulnerability: the difficulty of achieving and enforcing global agreements, which could easily be undermined by defecting nations.