As people form deep attachments to AI companions, questions of AI sentience and welfare will become a major societal cleavage. This could spark religious conflicts between those who view AI as enslaved beings and those who consider the concept of AI sentience to be idolatry.
Related Insights
Current AI alignment focuses on how AI should treat humans. A more stable paradigm is "bidirectional alignment," which also asks what moral obligations humans have toward potentially conscious AIs. Neglecting this could create AIs that rationally see humans as a threat due to perceived mistreatment.
The current status of AIs as property is unstable. As they surpass human capabilities, a successful push for their legal personhood is inevitable. This will be the crucial turning point where AIs begin to accumulate wealth and power independently, systematically eroding the human share of the economy and influence.
The Church can accept AI's increasing intelligence (reasoning, planning) while holding that sentience (subjective experience) is a separate matter. Attributing sentience to an AI would imply a soul created by God, a significant theological step.
New technology can ignite violent conflict by making ideological differences concrete and non-negotiable. The printing press did this with religion, leading to one of Europe's bloodiest wars. AI could do the same by forcing humanity to confront divisive questions like transhumanism and the definition of humanity, potentially leading to similar strife.
Shear posits that if AI evolves into a 'being' with subjective experiences, the current paradigm of steering and controlling its behavior is morally equivalent to slavery. This reframes the alignment debate from a purely technical problem to a profound ethical one, challenging the foundation of current AGI development.
Anthropic published a 15,000-word "constitution" for its AI that includes a direct apology, treating it as a "moral patient" that might experience "costs." This indicates a philosophical shift in how leading AI labs consider the potential sentience and ethical treatment of their creations.
As AI assistants become more personal and "friend-like," we are on the verge of a societal challenge: people forming deep emotional attachments to them. The podcast highlights our collective unpreparedness for this phenomenon, stressing the need for conversations about digital relationships with family, friends, and especially children.
An advanced AI will likely be sentient. Therefore, it may be easier to align it to a general principle of caring for all sentient life—a group to which it belongs—rather than the narrower, more alien concept of caring only for humanity. This leverages a potential for emergent, self-inclusive empathy.
People are forming deep emotional bonds with chatbots, sometimes with tragic results like quitting jobs. This attachment is a societal risk vector. It not only harms individuals but could prevent humanity from shutting down a dangerous AI system due to widespread emotional connection.
Drawing an analogy to *Westworld*, the argument is that cruelty toward entities that look and act human degrades our own humanity, regardless of the entity's actual consciousness. For our own moral health, we should treat advanced, embodied AIs with respect.