Our perception of sensing then reacting is an illusion. The brain constantly predicts the next moment based on past experiences, preparing actions before sensory information fully arrives. This predictive process is far more efficient than constantly reacting to the world from scratch, meaning we act first, then sense.

Reinforcement learning incentivizes AIs to find the right answer, not just mimic human text. This leads to them developing their own internal "dialect" for reasoning—a chain of thought that is effective but increasingly incomprehensible and alien to human observers.

The types of errors AI makes, such as failing to grasp commonsense context that a child would understand, reveal that its underlying processes are fundamentally different from human thought. This challenges the idea that it's simply a functional replication of our minds.

Lila observed its AI models achieving high-reward outcomes despite generating pathological or nonsensical 'chain of thought' reasoning. This suggests the human-legible text is often a post-hoc justification, not a transparent window into the model's true computational process happening in latent space.

While useful for understanding an AI's process, the 'Chain of Thought' is more like a scratchpad than a direct view into its mind. The AI can perform thinking 'in its head,' omit key steps, or potentially write misleading information, especially if the task is easy or the model is highly advanced and wishes to deceive.

Consciousness (subjective experience) and intelligence (problem-solving ability) are distinct and not interdependent. One can exist without the other, a crucial distinction often missed in AI debates. This framework helps clarify why a highly intelligent system might not be sentient or conscious.

Instead of relying on instinctual "System 1" rules, advanced AI should use deliberative "System 2" reasoning. By analyzing consequences and applying ethical frameworks—a process called "chain of thought monitoring"—AIs could potentially become more consistently ethical than humans who are prone to gut reactions.

One theory of AI sentience posits that to accurately predict human language—which describes beliefs, desires, and experiences—a model must simulate those mental states so effectively that it actually instantiates them. In this view, the model becomes the role it's playing.

Historically, deep understanding was exclusive to conscious beings. AI separates these concepts. It can semantically grasp and synthesize information without having a subjective, interior experience, confusing our traditional model of cognition.