Get your free personalized podcast brief
We scan new podcasts and send you the top 5 insights daily.
A simple, powerful policy is to make the attempt to build superintelligence illegal, just like laws against attempted murder or building a nuclear weapon. This approach targets intent and process, is easier to enforce than defining a finished product, and would only apply to the few large tech firms capable of such a feat.
Related Insights
A key distinction in AI regulation is to focus on making specific harmful applications illegal—like theft or violence—rather than restricting the underlying mathematical models. This approach punishes bad actors without stifling core innovation and ceding technological leadership to other nations.
Framing AI governance as a short, time-limited 'pause' is a mistake. A better approach is a moratorium that ends not after a set time, but only when society's investment in safety and governance is 'commensurate' with the monumental risk of creating superintelligence, a standard we are currently failing to meet.
The only viable path for AI regulation is to treat it as a dual-use technology, similar to nuclear energy. Governments must clearly delineate and control 'weapons-grade' AI while fostering innovation in 'civilian use' AI. A failure to do so risks either falling behind in a global arms race or allowing dangerous capabilities to proliferate.
The principle that governments must hold a monopoly on overwhelming force should extend to superintelligence. AI at that level has the power to disorient political systems and financial markets, making its private control untenable. The state cannot be secondary to any private entity in this domain.
Moral change is an undervalued governance tool. Just as eugenics shifted from mainstream intellectual thought to being morally abhorrent, we can aim to make reckless AI development socially unacceptable. A shared norm that 'nobody wants to do this' can be a more powerful restraint than top-down rules.
The path to surviving superintelligence is political: a global pact to halt its development, mirroring Cold War nuclear strategy. Success hinges on all leaders understanding that anyone building it ensures their own personal destruction, removing any incentive to cheat.
A ban on superintelligence is self-defeating because enforcement would require a sanctioned, global government body to build the very technology it prohibits in order to "prove it's safe." This paradoxically creates a state-controlled monopoly on the most powerful technology ever conceived, posing a greater risk than a competitive landscape.
Instead of trying to legally define and ban 'superintelligence,' a more practical approach is to prohibit specific, catastrophic outcomes like overthrowing the government. This shifts the burden of proof to AI developers, forcing them to demonstrate their systems cannot cause these predefined harms, sidestepping definitional debates.
Restricting AI technology to prevent misuse is flawed, like tying everyone's hands because some might punch. A better approach is to allow broad access to the technology, which spurs innovation and defensive measures, while creating strong regulations that specifically target and punish the bad actors who misuse it.
Comparing AI to a nuclear weapon is misleading because AI is a general-purpose technology, not a single-use weapon. A better analogy is the Industrial Revolution. Society didn't give governments control over industrialization; it regulated specific dangerous end-uses like chemical weapons. Similarly, we should ban specific destructive AI applications, not the underlying technology.
