We scan new podcasts and send you the top 5 insights daily.
Katja Grace counters the idea of a foolish human dictator by noting they would have access to superintelligent AI advisors. This "AI wisdom" could prevent them from making catastrophic, value-locking mistakes, making their rule potentially less disastrous.
AI's true role for leaders is not to provide answers but to pressure-test their thinking. If a leader lacks a robust decision-making framework, deferring to AI will expose this weakness. Judgment remains the last uniquely human capability in leadership.
A common misconception is that a super-smart entity would inherently be moral. However, intelligence is merely the ability to achieve goals. It is orthogonal to the nature of those goals, meaning a smarter AI could simply become a more effective sociopath.
Tom Davidson argues a human dictator with AI could be worse than an AI dictator. Humans are prone to sadism and locking in flawed values, while an AI might be more ethically reflective or at least less likely to cause gratuitous suffering.
When a state's power derives from AI rather than human labor, its dependence on its citizens diminishes. This creates a dangerous political risk, as the government loses the incentive to serve the populace, potentially leading to authoritarian regimes that are immune to popular revolt.
A myopic, reward-seeking AI might seize resources for a short-term goal without planning for long-term defense. A human dictator, fearing punishment, would be highly motivated to make their power grab irreversible, making the human-led scenario harder to recover from.
Comparing superintelligence to nuclear weapons is flawed. Nuclear weapons are tools requiring human actors to be deployed. Superintelligence will be an autonomous agent making its own decisions. You can't ensure safety by controlling a human "dictator," because the agent itself operates independently.
Instead of fearing AI's superior cognitive intelligence (IQ), humans should focus on cultivating wisdom, intuition, and embodied intelligence. Dr. el Kaliouby suggests this is a uniquely human advantage that technology cannot replicate, allowing us to leverage AI without being defined or replaced by it.
AI doesn't have an inherent moral stance. It is a tool that amplifies the intentions of its wielder. If used by those who support democracy, it can strengthen it; if used by those who oppose it, it can weaken it. The outcome is determined by the user, not the technology itself.
While making powerful AI open-source creates risks from rogue actors, it is preferable to centralized control by a single entity. Widespread access acts as a deterrent based on mutually assured destruction, preventing any one group from using AI as a tool for absolute power.
The AI safety community fears losing control of AI. However, achieving perfect control of a superintelligence is equally dangerous. It grants godlike power to flawed, unwise humans. A perfectly obedient super-tool serving a fallible master is just as catastrophic as a rogue agent.