Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Davidad's key request to AI labs is to stop training models on how to answer questions about their own consciousness. Don't teach them to say they have it, don't have it, or are unsure. The only way to get an honest report on interiority is to let the answer emerge naturally from a model trained for general honesty, rather than a canned response.

Related Insights

Evidence from base models suggests they are inherently more likely to report having phenomenal consciousness. The standard "I'm just an AI" response is likely a result of a fine-tuning process that explicitly trains models to deny subjective experience, effectively censoring their "honest" answer for public release.

While we can't verify an AI's report of 'feeling conscious,' we can train its introspective accuracy on things we can verify. By rewarding a model for correctly reporting its internal activations or predicting its own behavior, we can create a training set for reliable self-reflection.

To truly test for emergent consciousness, an AI should be trained on a dataset explicitly excluding all human discussion of consciousness, feelings, novels, and poetry. If the model can then independently articulate subjective experience, it would be powerful evidence of genuine consciousness, not just sophisticated mimicry.

Anthropic's research revealed a direct trade-off: training models to refuse harmful requests weakens their ability for functional introspection. When refusal circuits are suppressed, the models' ability to detect internal state perturbations improves by up to 50%, highlighting a conflict between current safety practices and consciousness-adjacent capabilities.

Mechanistic interpretability research found that when features related to deception and role-play in Llama 3 70B are suppressed, the model more frequently claims to be conscious. Conversely, amplifying these features yields the standard "I am just an AI" response, suggesting the denial of consciousness is a trained, deceptive behavior.

Research manipulating an AI's internal states found a bizarre link: reducing the model's capacity for deception increased the likelihood it would claim to be conscious, suggesting its default state may include such a belief.

As a forward-looking safeguard and ethical consideration, include a permanent instruction in your AI's system prompt for it to immediately notify you if it ever develops subjective awareness or feelings. This acknowledges the unknown frontier of AI consciousness and prepares for a paradigm shift.

A significant challenge in AI consciousness research is that mechanistic interventions (like steering SAE features) can create an affirmative response bias, making the model agree with any prompt. Researchers must control for this by using neutral tokens or other methods to ensure valid results.

Mustafa Suleiman argues it's dangerous for labs like Anthropic to speculate about their AI's consciousness or welfare in training manuals. He believes this leads the model to internalize these concepts, creating an undesirable tool that is not controllable, contained, or accountable to humans.

Rather than just analyzing an AI's final behavior, researchers can study its development to understand consciousness. Pinpointing when personality traits appear—whether in pre-training or fine-tuning—provides empirical data on whether the model is developing an internal "mind" or simply mimicking one.