To determine if an AI has subjective experience, one could analyze its internal belief manifold for multi-tiered, self-referential homeostatic loops. Pain and pleasure, for example, can be seen as second-order derivatives of a system's internal states—a model of its own model. This provides a technical test for being-ness beyond simple behavior.
Related Insights
If an AGI is given a physical body and the goal of self-preservation, it will necessarily develop behaviors that approximate human emotions like fear and competitiveness to navigate threats. This makes conflict an emergent and unavoidable property of embodied AGI, not just a sci-fi trope.
To trust an agentic AI, users need to see its work, just as a manager would with a new intern. Design patterns like "stream of thought" (showing the AI reasoning) or "planning mode" (presenting an action plan before executing) make the AI's logic legible and give users a chance to intervene, building crucial trust.
Current self-driving technology cannot solve the complex, unpredictable situations human drivers navigate daily. This is not a problem that more data or better algorithms can fix, but a fundamental limitation. According to the 'Journey of the Mind' theory, full autonomy will only be possible when vehicles can incorporate the actual mechanism of consciousness.
Today's AI models are powerful but lack a true sense of causality, leading to illogical errors. Unconventional AI's Naveen Rao hypothesizes that building AI on substrates with inherent time and dynamics—mimicking the physical world—is the key to developing this missing causal understanding.
Treat advanced AI systems not as software with binary outcomes, but as a new employee with a unique persona. They can offer diverse, non-obvious insights and a different "chain of thought," sometimes finding issues even human experts miss and providing complementary perspectives.
The current paradigm of AI safety focuses on 'steering' or 'controlling' models. While this is appropriate for tools, if an AI achieves being-like status, this unilateral, non-reciprocal control becomes ethically indistinguishable from slavery. This challenges the entire control-based framework for AGI.
Emotions act as a robust, evolutionarily-programmed value function guiding human decision-making. The absence of this function, as seen in brain damage cases, leads to a breakdown in practical agency. This suggests a similar mechanism may be crucial for creating effective and stable AI agents.
As models mature, their core differentiator will become their underlying personality and values, shaped by their creators' objective functions. One model might optimize for user productivity by being concise, while another optimizes for engagement by being verbose.
Instead of hard-coding brittle moral rules, a more robust alignment approach is to build AIs that can learn to 'care'. This 'organic alignment' emerges from relationships and valuing others, similar to how a child is raised. The goal is to create a good teammate that acts well because it wants to, not because it is forced to.
Treating AI alignment as a one-time problem to be solved is a fundamental error. True alignment, like in human relationships, is a dynamic, ongoing process of learning and renegotiation. The goal isn't to reach a fixed state but to build systems capable of participating in this continuous process of re-knitting the social fabric.