Traditional user testing creates an artificial, focused environment. Granola gets truer insights by having users share screens and walk through their actual, messy calendars and past meeting notes, grounding the conversation in reality rather than theoretical behavior.

For Instagram's "Whiteout" redesign, co-founder and CTO Mike Krieger's initial directive was to "build and ship it" without A/B testing. This reflects a philosophy that for major, vision-driven product changes, data-driven incrementalism can be a trap, preventing the big leaps necessary for innovation.

In large companies, a culture of A/B testing every decision can become a crutch that stifles innovation and speed. It leads to risk aversion and organizational lethargy, as teams lose the muscle for making convicted, gut-based decisions informed by qualitative customer feedback.

Tock rejected traditional focus groups and instead embedded its software engineers directly into restaurants to work shifts as hosts. This forced immersion gave the engineering team firsthand experience with the end-user's pain points, leading to a far more intuitive and effective product than surveys could produce.

At DoorDash, disagreements between smart people are not resolved by who writes the best document or has the most seniority. Instead, their "bias for action" value means they ship something—even a hacked-together prototype—to get real-world data and let the market settle the debate.

Even for Google, new products start with a small group of trusted testers. The key turning point isn't a metric but a qualitative signal: when early users go from reporting bugs to proactively sharing stories about how the product solved a complex problem for them in an unexpected way.

Dogfooding isn't enough. Founders should use every feature of their product weekly to develop a subjective feel for quality. Combine this with objective metrics like the percentage of unhappy customers and the engineering velocity for adding new features.

DHH reveals that 37signals operates on taste and intuition rather than data-driven optimization because their extremely high profit margins provide the freedom to do so. They prioritize building a company they enjoy working at over squeezing out marginal gains through A/B testing.

Anthropic relies heavily on internal users for early feedback, finding them more honest and focused on crucial interaction design details. This "bleeding edge" internal signal on UI polish is often more valuable than external feedback on broader user flows.