We scan new podcasts and send you the top 5 insights daily.
Founders with low trial volume often mistakenly try to A/B test small changes. With insufficient data, such tests are meaningless. Instead, they should focus on making big, obvious improvements based on gut feel and qualitative feedback. At this stage, the goal isn't optimization; it's finding significant wins that don't require statistical validation.
Artist's co-founder warns that the biggest mistake founders make is building technology too early. Her team validated their text-based learning concept by manually texting early users, confirming the core hypothesis and user engagement before committing significant engineering resources.
The goal of early validation is not to confirm your genius, but to risk being proven wrong before committing resources. Negative feedback is a valuable outcome that prevents building the wrong product. It often reveals that the real opportunity is "a degree to the left" of the original idea.
The job of an early founder isn't to be right, but to discover the truth about the market. This requires shipping imperfect products quickly to test assumptions, gathering harsh feedback, and being humble enough to accept when you are wrong.
Knowing when and how to pivot isn't a data-driven process. It's a messy decision made with incomplete information when the current path is failing. Early customers often provide contradictory feedback, meaning the founder must rely on their intuition and a small circle of trusted advisors to choose the new direction.
As articulated by Eric Ries in 'The Lean Startup,' raw speed of shipping is meaningless if you're building in the wrong direction. The true measure of progress is how quickly a team can validate assumptions and learn what customers want, which prevents costly rework.
Don't attempt traditional A/B testing on a low-traffic website; the results will be statistically invalid. Instead, use qualitative user testing methods like preference tests. This approach provides directional data to guide decisions, which is far more reliable than guesswork or a flawed A/B test.
Early demos shouldn't be used to ask, "Did we build the right thing?" Instead, present them to customers to test your core assumptions and ask, "Did we understand your problem correctly?" This reframes feedback, focusing on the root cause before investing heavily in a specific solution.
Instead of immediately launching expensive A/B tests or ad campaigns, first validate your messaging qualitatively. Put it in front of a panel of ideal customers and ask open-ended questions to get faster, richer feedback on clarity and resonance.
Instead of only testing minor changes on a finished product, like button color, use A/B testing early in the development process. This allows you to validate broad behavioral science principles, such as social proof, for your specific challenge before committing to a full build.
Pincus critiques the 'MVP trap,' where teams waste time building a product based on a flawed premise. He advocates for a 'failure machine' that rapidly tests many raw ideas (e.g., click-through rates on mockups) to find what users actually want before committing engineering resources.