When asked to describe a user process, an LLM provides the textbook version. It misses the real-world chaos—forgotten tasks, interruptions, and workarounds. These messy details, which only emerge from talking to real people, are where the most valuable product opportunities are found.

Related Insights

Current LLMs are intelligent enough for many tasks but fail because they lack access to complete context—emails, Slack messages, past data. The next step is building products that ingest this real-world context, making it available for the model to act upon.

Asking users for solutions yields incremental ideas like "faster horses." Instead, ask them to tell detailed stories about their workflow. This narrative approach uncovers the true context, pain points, and decision journeys that direct questions miss, leading to breakthrough insights about the actual problem to be solved.

Many teams wrongly focus on the latest models and frameworks. True improvement comes from classic product development: talking to users, preparing better data, optimizing workflows, and writing better prompts.

To get unbiased user feedback, avoid asking leading questions like "What are your main problems?" Instead, prompt users to walk you through their typical workflow. In describing their process, they will naturally reveal the genuine friction points and hacks they use, providing much richer insight than direct questioning.

Customers describe an idealized version of their world in interviews. To understand their true problems and workflows, you must be physically present. This uncovers the crucial gap between their perception and day-to-day reality.

AI tools can handle administrative and analytical tasks for product managers, like summarizing notes or drafting stories. However, they lack the essential human elements of empathy, nuanced judgment, and creativity required to truly understand user problems and make difficult trade-off decisions.

While AI efficiently transcribes user interviews, true customer insight comes from ethnographic research—observing users in their natural environment. What people say is often different from their actual behavior. Don't let AI tools create a false sense of understanding that replaces direct observation.

Users often develop multi-product workarounds for issues they don't even recognize as solvable problems. Identifying these subconscious behaviors reveals significant innovation opportunities that users themselves cannot articulate.

Developers often test AI systems with well-formed, correctly spelled questions. However, real users submit vague, typo-ridden, and ambiguous prompts. Directly analyzing these raw logs is the most crucial first step to understanding how your product fails in the real world and where to focus quality improvements.

Chatbots are fundamentally linear, which is ill-suited for complex tasks like planning a trip. The next generation of AI products will use AI as a co-creation tool within a more flexible canvas-like interface, allowing users to manipulate and organize AI-generated content non-linearly.