The property rights argument for AI safety hinges on an ecosystem of multiple, interdependent AIs. The strategy breaks down in a scenario where a single AI achieves a rapid, godlike intelligence explosion. Such an entity would be self-sufficient and could expropriate everyone else without consequence, as it wouldn't need to uphold the system.
Related Insights
Coined in 1965, the "intelligence explosion" describes a runaway feedback loop. An AI capable of conducting AI research could use its intelligence to improve itself. This newly enhanced intelligence would make it even better at AI research, leading to exponential, uncontrollable growth in capability. This "fast takeoff" could leave humanity far behind in a very short period.
Granting AIs property rights incentivizes them to uphold the system that protects those rights. This makes them less likely to engage in actions like expropriating human property or committing genocide, as such actions would destabilize the very system that secures their own wealth and agency.
While mitigating catastrophic AI risks is critical, the argument for safety can be used to justify placing powerful AI exclusively in the hands of a few actors. This centralization, intended to prevent misuse, simultaneously creates the monopolistic conditions for the Intelligence Curse to take hold.
Early AIs can be kept safe via direct alignment. However, as AIs evolve and "value drift" occurs, this technical safety could fail. A pre-established economic and political system based on property rights can then serve as the new, more robust backstop for ensuring long-term human safety.
Debates about AI and inequality often assume today's financial institutions will persist. However, in a fast takeoff scenario with superintelligence, concepts like property rights and stock certificates might become meaningless as new, unimaginable economic and political systems emerge.
Instead of building a single, monolithic AGI, the "Comprehensive AI Services" model suggests safety comes from creating a buffered ecosystem of specialized AIs. These agents can be superhuman within their domain (e.g., protein folding) but are fundamentally limited, preventing runaway, uncontrollable intelligence.
A system where AIs have property rights creates a powerful economic disincentive to build unaligned AIs. If a company cannot reliably align an AI to remit its wages, the massive development cost becomes a loss. This framework naturally discourages the creation of potentially dangerous, uncooperative models.
AI safety scenarios often miss the socio-political dimension. A superintelligence's greatest threat isn't direct action, but its ability to recruit a massive human following to defend it and enact its will. This makes simple containment measures like 'unplugging it' socially and physically impossible, as humans would protect their new 'leader'.
The fundamental challenge of creating safe AGI is not about specific failure modes but about grappling with the immense power such a system will wield. The difficulty in truly imagining and 'feeling' this future power is a major obstacle for researchers and the public, hindering proactive safety measures. The core problem is simply 'the power.'
A more likely AI future involves an ecosystem of specialized agents, each mastering a specific domain (e.g., physical vs. digital worlds), rather than a single, monolithic AGI that understands everything. These agents will require protocols to interact.
