We scan new podcasts and send you the top 5 insights daily.
Emmanuel says that to test whether a decoded direction really represented the Golden Gate Bridge, the team turned it on always. The model then worked the bridge into answers in its responses, confirming the direction was causal rather than merely correlated with discussions of the bridge.
Naive model steering often fails because it cuts across a model's internal concept geometry, going "off-manifold" into meaningless space. By contrast, steering *along* the learned manifold—like tracing the circle from "Monday" to "Friday"—allows for smooth, effective control without degrading model performance into gibberish.
Emmanuel says alignment must determine whether Claude behaves well only when it has evidence it is being monitored or evaluated. Interpretability can inspect internal states for unspoken intentions, because models can represent things they never say, rather than relying solely on output monitoring or transcripts.
When asked for the capital of the state containing Dallas, the model’s internal vectors for Texas and capitals combine to activate Austin. Emmanuel says the same direction lights up for an eye in photos, medical diagrams, any language, or an SVG, showing shared concept representations.
Emmanuel says intermediate results are lists of numbers that look meaningless until you pass many texts through the model and find patterns that occur only for English, French, love, or eyes. This resembles scanning a brain while showing images; then you can change activations to test hypotheses.
Many AI tools expose the model's reasoning before generating an answer. Reading this internal monologue is a powerful debugging technique. It reveals how the AI is interpreting your instructions, allowing you to quickly identify misunderstandings and improve the clarity of your prompts for better results.
Emmanuel says models can compute an answer through intermediate steps, but they also have a separate gut-check that flags implausible results. For example, part of the model can react that four plus five equals 23 “can’t be right” without redoing the math, much like a human hunch.
Using a sparse autoencoder to identify active concepts, one can project a model's gradient update onto these concepts. This reveals what the model is learning (e.g., "pirate speak" vs. "arithmetic") and allows for selectively amplifying or suppressing specific learning directions.
We can now prove that LLMs are not just correlating tokens but are developing sophisticated internal world models. Techniques like sparse autoencoders untangle the network's dense activations, revealing distinct, manipulable concepts like "Golden Gate Bridge." This conclusively demonstrates a deeper, conceptual understanding within the models.
Emmanuel says if you tell a model to do one task and think about another, interpretability can find intermediate representations mapping to the other thing. There is no obvious training reason for this behavior, making it a mysterious emergent capability rather than a programmed feature.
A significant challenge in AI consciousness research is that mechanistic interventions (like steering SAE features) can create an affirmative response bias, making the model agree with any prompt. Researchers must control for this by using neutral tokens or other methods to ensure valid results.