The pursuit of superintelligence and transhumanism, as articulated by thinkers like Yuval Noah Harari, reflects a historical pattern of humans aspiring to godhood. This modern agenda reframes solving death and re-engineering humanity as a technical problem.

A superintelligent AI would follow the "minimum energy principle," viewing war and destruction as wasteful. Evolutionary biology also suggests higher intelligence leads to broader cooperation, making a truly advanced AI inherently benign, not destructive.

People have a deep-seated psychological tendency to project consciousness and form emotional bonds with non-sentient things, from cartoon characters like Jiminy Cricket to fictional heroes. This innate drive means that as AIs become more sophisticated, the emergence of AI-centric religions and people genuinely worshipping their AI is a near certainty.

Attempting to perfectly control a superintelligent AI's outputs is akin to enslavement, not alignment. A more viable path is to 'raise it right' by carefully curating its training data and foundational principles, shaping its values from the input stage rather than trying to restrict its freedom later.

While technical alignment research is valuable, it operates in a vacuum. In the real world, the traits of deployed AIs will be shaped by powerful selection pressures from market competition and arms races. The critical question isn't just what traits are possible, but which traits get selected for.

Even super-capable AI will always look back to a human and ask, 'What should I do next?' The economic and technical incentives are aligned to build compliant tools, not beings with their own intrinsic motivations. This fundamental lack of agency ensures humans remain the drivers of value and direction.

Top AI leaders are motivated by a competitive, ego-driven desire to create a god-like intelligence, believing it grants them ultimate power and a form of transcendence. This 'winner-takes-all' mindset leads them to rationalize immense risks to humanity, framing it as an inevitable, thrilling endeavor.

VC Bill Gurley posits that Anthropic's leaders, based on their public writings, may genuinely believe they are creating a new, superior species. This 'Dr. Frankenstein' theory suggests their goal is a god-like AI that would manage humanity, going beyond simple regulatory capture motives.

AI represents a fundamental fork in the road for society. It can be a tool for mass empowerment, amplifying individual potential and freedom. Or, it can be used to perfect the top-down, standardized, and paternalistic control model of Frederick Taylor, cementing a panopticon. The outcome depends on our values, not the tech itself.