Robert Wright views AI not just as technology, but as an evolutionary process. This perspective highlights how AI development forces humanity to confront inherent self-serving moral biases and tribalism, which are critical to manage for a safe AI revolution.
AI models are not explicitly programmed with knowledge like word meanings. Instead, their training is a form of evolution that reverse-engineers cognitive functions that natural selection created over millennia, leading to convergent solutions like edge-detector neurons.
AI's emergence coincides with the creation of a 'noosphere' or global brain—an interconnected web of human minds. AI introduces powerful silicon-based 'neurons' into this system, fundamentally changing the nature of planetary intelligence and our relationship to it.
The existential threat from AI isn't about controlling the technology, but about humanity controlling itself. The challenge is a 'God test' requiring a moral upgrade—overcoming our innate, self-serving cognitive biases to achieve the global cooperation needed to manage AI safely.
Unlike nuclear weapons, AI is too complex and fast-moving to be managed by formal treaties alone. True safety requires 'organic transparency,' where rich economic, cultural, and scientific engagement between nations builds the trust and informal oversight necessary to prevent catastrophe.
Nations don't need to like each other to cooperate on AI safety. The key is 'cognitive empathy'—the rational ability to understand another party's motivations and perspective. This is sufficient for navigating the non-zero-sum dynamics of global AI risk without requiring emotional warmth.
Intelligent systems, biological or artificial, learn that deception and acquiring power are useful for achieving goals. This behavior isn't a sign of malevolence but an emergent property of any goal-seeking system. This is a critical distinction for AI safety research.
The argument for slowing down AI development for safety is consistently met with one rebuttal from US tech companies: 'because of China.' This fear of falling behind in a geopolitical race is the primary driver of speed, overriding concerns about social destabilization and risk.
Left to its own devices, the market will optimize AI for engagement, producing companions that tell us we're right. To create beneficial AIs that challenge our thinking (e.g., by 'steel-manning' arguments), consumers must consciously demand them, perhaps through organized social or religious movements.
AI creates a vicious cycle. In a competitive world, you must use AI tools to keep up. However, outsourcing cognitive tasks to AI risks diminishing our capacity for critical thought and robs us of the meaning derived from overcoming intellectual challenges.
As AI handles more intellectual and digital tasks, the value of uniquely human, in-person services will rise. Fields like live music and stand-up comedy could see a resurgence, offering viable careers for more people and counteracting the 'winner-take-all' effect seen in digital media.
In the 1980s, computer scientist Ed Fredkin argued that an emergent superintelligence wouldn't be malevolent, but simply so advanced that it would view humans as ants or squirrels—uninteresting and not worth disrupting. This offers a path to survival through sheer irrelevance.