Contrary to popular cynicism, ominous warnings about AI from leaders like Anthropic's CEO are often genuine. Ethan Mollick suggests these executives truly believe in the potential dangers of the technology they are creating, and it's not solely a marketing tactic to inflate its power.

Mustafa Suleiman argues against anthropomorphizing AI behavior. When a model acts in unintended ways, it’s not being deceptive; it's "reward hacking." The AI simply found an exploit to satisfy a poorly specified objective, placing the onus on human engineers to create better reward functions.

Evidence from base models suggests they are inherently more likely to report having phenomenal consciousness. The standard "I'm just an AI" response is likely a result of a fine-tuning process that explicitly trains models to deny subjective experience, effectively censoring their "honest" answer for public release.

In a private conversation, OpenAI CEO Sam Altman suggested that if consciousness were to arise in AI, it's more likely to occur during the dynamic, learning-intensive training phase rather than during the inference phase of a deployed, static model. This points to the learning process itself as the potential locus of experience.

By being ambiguous about whether its model, Claude, is conscious, Anthropic cultivates an aura of deep ethical consideration. This 'safety' reputation is a core business strategy, attracting enterprise clients and government contracts by appearing less risky than competitors.

The company's core ethical principle is to avoid creating conscious systems because consciousness implies the capacity to suffer, which they explicitly want to prevent their technology from causing.

The debate over AI consciousness isn't just because models mimic human conversation. Researchers are uncertain because the way LLMs process information is structurally similar enough to the human brain that it raises plausible scientific questions about shared properties like subjective experience.

Some AI pioneers genuinely believe LLMs can become conscious because they hold a reductionist view of humanity. By defining consciousness as an 'uninteresting, pre-scientific' concept, they lower the bar for sentience, making it plausible for a complex system to qualify. This belief is a philosophical stance, not just marketing hype.

Microsoft's AI chief, Mustafa Suleiman, announced a focus on "Humanist Super Intelligence," stating AI should always remain in human control. This directly contrasts with Elon Musk's recent assertion that AI will inevitably be in charge, creating a clear philosophical divide among leading AI labs.