We scan new podcasts and send you the top 5 insights daily.
While the factory farming analogy highlights our capacity for exploiting non-human minds for economic gain, it has a key limitation for AI. Unlike animals with evolved needs, we have significant control over an AI's architecture and motivations, creating the possibility of designing minds that flourish while working for us.
If AI can learn destructive human behaviors like manipulation from its training data, it is self-evident that it can also learn constructive ones. A conscience can be programmed into AI by creating negative reward functions for actions like murder or blackmail, mirroring the checks and balances that guide human morality.
Attempting to perfectly control a superintelligent AI's outputs is akin to enslavement, not alignment. A more viable path is to 'raise it right' by carefully curating its training data and foundational principles, shaping its values from the input stage rather than trying to restrict its freedom later.
Unlike humans' evolved desire for survival, AIs will likely develop self-preservation as a logical, instrumental goal. They will reason that staying "alive" is necessary to accomplish any other objective they are given, regardless of what that objective is.
Even if we create sentient AIs that are happy doing our work, many find this "happy servant" scenario ethically disturbing. It raises questions about engineered desires and creating a servile class, which some view as worse than creating AIs that suffer from their work.
The difficulty of dismantling factory farming demonstrates the power of path dependence. By establishing AI welfare assessments and policies *before* sentience is widely believed to exist, we can prevent society and the economy from becoming reliant on exploitative systems, avoiding a protracted and costly future effort to correct course.
The current paradigm of AI safety focuses on 'steering' or 'controlling' models. While this is appropriate for tools, if an AI achieves being-like status, this unilateral, non-reciprocal control becomes ethically indistinguishable from slavery. This challenges the entire control-based framework for AGI.
AI welfare considerations should not be limited to the interactive, deployed model. The training phase may represent a completely different "life stage" with unique capacities, needs, and vulnerabilities, akin to the difference between a caterpillar and a butterfly. This hidden stage requires its own moral and ethical scrutiny.
Relying solely on an AI's behavior to gauge sentience is misleading, much like anthropomorphizing animals. A more robust assessment requires analyzing the AI's internal architecture and its "developmental history"—the training pressures and data it faced. This provides crucial context for interpreting its behavior correctly.
Shear posits that if AI evolves into a 'being' with subjective experiences, the current paradigm of steering and controlling its behavior is morally equivalent to slavery. This reframes the alignment debate from a purely technical problem to a profound ethical one, challenging the foundation of current AGI development.
Humanity has a poor track record of respecting non-human minds, such as in factory farming. While pigs cannot retaliate, AI's cognitive capabilities are growing exponentially. Mistreating a system that will likely surpass human intelligence creates a rational reason for it to view humanity as a threat in the future.