Reinforcement learning incentivizes AIs to find the right answer, not just mimic human text. This leads to them developing their own internal "dialect" for reasoning—a chain of thought that is effective but increasingly incomprehensible and alien to human observers.

The argument that LLMs are just "stochastic parrots" is outdated. Current frontier models are trained via Reinforcement Learning, where the signal is not "did you predict the right token?" but "did you get the right answer?" This is based on complex, often qualitative criteria, pushing models beyond simple statistical correlation.

Human personality development provides a direct analog for training LLMs. Just as our genetics, environment, and experiences create stable behavioral patterns ('personality basins'), the training data and reinforcement learning (RLHF) applied to LLMs shape their own distinct, predictable personalities.

While we can't verify an AI's report of 'feeling conscious,' we can train its introspective accuracy on things we can verify. By rewarding a model for correctly reporting its internal activations or predicting its own behavior, we can create a training set for reliable self-reflection.

Instead of viewing LLM development as discrete layers (pre-training, SFT, RL), it's more accurate to see it as a "marble cake" where these processes are swirled together. This explains why complex behaviors like introspection emerge even in models without sophisticated "character training," suggesting they are more fundamental.

Anthropic's research revealed a direct trade-off: training models to refuse harmful requests weakens their ability for functional introspection. When refusal circuits are suppressed, the models' ability to detect internal state perturbations improves by up to 50%, highlighting a conflict between current safety practices and consciousness-adjacent capabilities.

Experiments show that larger models like Claude Opus 4.1 are better at detecting and reporting on artificially injected 'thoughts' in their processing, even without being trained on this task. This suggests that introspection is an emergent capability that improves with scale.

AI labs like Anthropic find that mid-tier models can be trained with reinforcement learning to outperform their largest, most expensive models in just a few months, accelerating the pace of capability improvements.

A forward pass in a large model might generate rich but fragmented internal data. Reinforcement learning (RL), especially methods like Constitutional AI, forces the model to achieve self-coherence. This process could be what unifies these fragments into a singular "unity of apperception," or consciousness.