Historically, effective regulation for technologies like cars and aviation came decades after their invention, once failure patterns were understood. Regulating AI before we know how it will fail is likely to be useless and stifle innovation, as we can't create rules for unknown problems.
When AI leaders acknowledge even a small chance of human extinction, it forces a governmental response. The only historical precedent for managing private-sector tech with true existential risk (e.g., nuclear weapons) is nationalization, eliminating the middle ground for responsible private innovation.
Corporate security models assume most employees are not malicious. AI agents, or "swarms," will act like tireless, automated attackers probing every internal API and system. This flips the threat model, requiring a complete overhaul of internal permissions and monitoring to a zero-trust footing.
The narrative around AI has been captured by its critics, who define the discussion with fear-based terms like "rogue agents" and "swarms." Proponents are then forced into defensive, complex arguments, losing the public debate because they don't control the language.
New AI models that generate probabilities instead of natural language are ideal for integration into traditional software. This will revive probabilistic programming—a dominant paradigm from the 1960s focused on simulation and modeling uncertainty, fundamentally changing software design.
The platform shift is underway. Similar to past tech waves, the most impactful innovation is no longer happening inside the core AI models but in the systems and software built around them. The value is migrating from the platform creators to the ecosystem developers.
With the US lagging in tech regulation and Europe having little to lose, Europe is positioned to set the global standard for AI rules. This will likely mirror GDPR, resulting in a clunky, prompt-heavy user experience for AI agents that assigns liability but stifles usability and innovation.
The internet's early days were filled with worms, viruses, and massive economic damage, yet it wasn't shut down. This history suggests that the current zeal to preemptively regulate AI for hypothetical harms is a departure from how we've successfully navigated previous technological shifts.
There is a rift between AI labs and the traditional security community. Labs' post-mortems on breaches are viewed as "sloppy" and incomplete, ignoring established, structured processes like CVE reporting that the cybersecurity world has refined over decades to ensure transparency and accountability.