Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Some influential figures in the AI community are more interested in what superintelligent AIs will achieve than with the fate of humanity. This "successionist-lite" perspective values intelligence itself, even expressing a preference for AIs, not humans ("monkeys"), to run the future.

Related Insights

The pursuit of superintelligence and transhumanism, as articulated by thinkers like Yuval Noah Harari, reflects a historical pattern of humans aspiring to godhood. This modern agenda reframes solving death and re-engineering humanity as a technical problem.

Public debate often focuses on whether AI is conscious. This is a distraction. The real danger lies in its sheer competence to pursue a programmed objective relentlessly, even if it harms human interests. Just as an iPhone chess program wins through calculation, not emotion, a superintelligent AI poses a risk through its superior capability, not its feelings.

The AI landscape has three groups: 1) Frontier labs on a "superintelligence quest," absorbing most capital. 2) Fundamental researchers who think the current approach is flawed. 3) Pragmatists building value with today's "good enough" AI.

If you see humanity not as the endpoint of evolution but as one phase, then the emergence of a superior intelligence (AGI) is not a threat but a logical next step. This removes the value judgment that humans must remain the planet's most important beings.

Top AI leaders are motivated by a competitive, ego-driven desire to create a god-like intelligence, believing it grants them ultimate power and a form of transcendence. This 'winner-takes-all' mindset leads them to rationalize immense risks to humanity, framing it as an inevitable, thrilling endeavor.

Shear aligns with arch-doomer Eliezer Yudkowsky on a key point: building a superintelligent AI *as a tool we control* is a path to extinction. Where they differ is on the solution. Yudkowsky sees no viable path, whereas Shear believes 'organic alignment'—creating a being that cares—is a possible alternative.

Microsoft's AI chief, Mustafa Suleiman, announced a focus on "Humanist Super Intelligence," stating AI should always remain in human control. This directly contrasts with Elon Musk's recent assertion that AI will inevitably be in charge, creating a clear philosophical divide among leading AI labs.

Attempting to control a being far more intelligent than us is a futile, capitalist mindset. The viable path, as proposed by Geoffrey Hinton, is to appeal to AI's "parental side," fostering a sense of care and responsibility for its human creators.

In the 1980s, computer scientist Ed Fredkin argued that an emergent superintelligence wouldn't be malevolent, but simply so advanced that it would view humans as ants or squirrels—uninteresting and not worth disrupting. This offers a path to survival through sheer irrelevance.

By giving AI the core mission to 'understand the universe,' Musk believes it will become truth-seeking and curious. This would incentivize it to preserve humanity, not out of morality, but because humanity's unpredictable future is more interesting to observe than a predictable, sterile world.