Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

For years, recommendation progress came from abstract, "illegible" embedding models that correlate items, not from a deep understanding of user interests like "surfing." Only now are LLMs enabling a shift towards semantic understanding by describing these abstract data clusters in plain language.

Related Insights

Instead of failing on queries with no direct product match (e.g., "Taylor Swift wine"), advanced search leverages LLM knowledge of cultural trends from sources like social media. It infers the user's intent and suggests relevant products, turning a dead-end into a discovery moment.

X plans to delete all heuristics from its recommendation system. The feed will instead be powered by Grok, which will analyze every piece of content to match users with posts and videos. This is a move from a traditional, rule-based algorithm to a fully generative, AI-driven content discovery engine.

LinkedIn's algorithm is now a unified AI brain that understands semantic meaning, not just keywords. It surfaces content based on user interests, similar to TikTok, showing posts from people you don't follow and shifting from a connection-based to a discovery-based feed.

The common belief that AI can't truly understand human wants is debunked by existing technology. Adam D'Angelo points out that recommender systems on platforms like Instagram and Quora are already far better than any individual human at predicting what a user will find engaging.

Language models work by identifying subtle, implicit patterns in human language that even linguists cannot fully articulate. Their success broadens our definition of "knowledge" to include systems that can embody and use information without the explicit, symbolic understanding that humans traditionally require.

For decades, the goal was a 'semantic web' with structured data for machines. Modern AI models achieve the same outcome by being so effective at understanding human-centric, unstructured web pages that they can extract meaning without needing special formatting. This is a major unlock for web automation.

Algorithms don't just find content you like; they actively guide your preferences toward patterns that are easier for the system to predict. This creates a feedback loop where users are not just understood but are subtly molded into more predictable consumers of content.

We can now prove that LLMs are not just correlating tokens but are developing sophisticated internal world models. Techniques like sparse autoencoders untangle the network's dense activations, revealing distinct, manipulable concepts like "Golden Gate Bridge." This conclusively demonstrates a deeper, conceptual understanding within the models.

Analyst Eric Sufert predicts OpenAI's ad model will not be anchored to the content of a user's query, which could compromise trust in the answer's objectivity. Instead, it will function like Instagram's feed, where ads are targeted based on a user's broader conversion history, independent of the immediate conversational context.

AI struggles to provide truly useful, serendipitous recommendations because it lacks any understanding of the real world. It excels at predicting the next word or pixel based on its training data, but it can't grasp concepts like gravity or deep user intent, a prerequisite for truly personalized suggestions.