Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

When asked for the capital of the state containing Dallas, the model’s internal vectors for Texas and capitals combine to activate Austin. Emmanuel says the same direction lights up for an eye in photos, medical diagrams, any language, or an SVG, showing shared concept representations.

Related Insights

Google's Embedding 2 model is a significant infrastructure upgrade because it is 'natively multimodal.' This allows AI to directly understand and retrieve images, diagrams, and text without first converting non-text data into lossy captions. This makes internal knowledge bases and co-pilots dramatically more effective and accurate for enterprises.

Goodfire's research on 'neural geometry' reveals that concepts inside models have distinct, low-dimensional shapes (manifolds). For example, numbers form a helix and temperature forms a spiral arc. Understanding these shapes allows for more precise and effective interventions, moving beyond linear vector manipulations.

Emmanuel says intermediate results are lists of numbers that look meaningless until you pass many texts through the model and find patterns that occur only for English, French, love, or eyes. This resembles scanning a brain while showing images; then you can change activations to test hypotheses.

Emmanuel says that to test whether a decoded direction really represented the Golden Gate Bridge, the team turned it on always. The model then worked the bridge into answers in its responses, confirming the direction was causal rather than merely correlated with discussions of the bridge.

The geometric similarities between how AI models and the brain represent concepts are a strong signal that both fields are converging on a fundamental truth. This alignment provides a practical tool for BCI development and a deeper understanding of intelligence itself, suggesting it's like a law of physics.

Moving beyond the simple Linear Representation Hypothesis, models organize concepts within sparse mixtures of subspaces. The specific geometry of these "manifolds" (e.g., a circle for days of the week) encodes the relationships and valid operations between concepts, like chemistry emerging from the periodic table.

We can now prove that LLMs are not just correlating tokens but are developing sophisticated internal world models. Techniques like sparse autoencoders untangle the network's dense activations, revealing distinct, manipulable concepts like "Golden Gate Bridge." This conclusively demonstrates a deeper, conceptual understanding within the models.

RAG systems are limited to direct retrieval and can't make spontaneous, abstract connections. This human-like ability to notice related but unasked-for concepts can only emerge from knowledge internalized within model weights, forming an associative memory.

Emmanuel says if you tell a model to do one task and think about another, interpretability can find intermediate representations mapping to the other thing. There is no obvious training reason for this behavior, making it a mysterious emergent capability rather than a programmed feature.

The ability of a single encoder to excel at both understanding and generating images indicates these two tasks are not as distinct as they seem. It suggests they rely on a shared, fundamental structure of visual information that can be captured in one unified representation.