In the debate between data-driven AB testing and intuitive 'taste' for product design, a humorous but practical career tip emerged: run the AB test to find the optimal solution (e.g., a blue button). Then, instead of presenting the data, confidently tell leadership the choice was based on your superior 'taste,' thereby building a reputation for invaluable intuition.
Related Insights
Marketing decisions should not be based on internal team members' subjective preferences, such as "I wouldn't click on that." Your team is not your target audience. A culture of A/B testing ideas should always take precedence over personal opinions to avoid a bad marketing environment.
Product leaders often feel they must present a perfect, unassailable plan to executives. However, the goal should be to start a discussion. Presenting an idea as an educated guess allows for a collaborative debate where you can gather more information and adjust the strategy based on leadership's feedback.
For Instagram's "Whiteout" redesign, co-founder and CTO Mike Krieger's initial directive was to "build and ship it" without A/B testing. This reflects a philosophy that for major, vision-driven product changes, data-driven incrementalism can be a trap, preventing the big leaps necessary for innovation.
Don't commit to a rigid career plan. Instead, treat your career like a product. Run small-scale experiments or 'MVPs'—like a 20% project, a volunteer role, or a teaching gig—to test your interest and aptitude for new skills before making a full commitment, then iterate based on the results.
In large companies, a culture of A/B testing every decision can become a crutch that stifles innovation and speed. It leads to risk aversion and organizational lethargy, as teams lose the muscle for making convicted, gut-based decisions informed by qualitative customer feedback.
As a junior IC at Instagram, Adrian was told leadership had "bigger fish to fry" than his A/B testing idea. He built a scrappy, functional prototype anyway, recruiting a PM for air cover. This bottoms-up initiative proved its value and ultimately led to his first senior promotion.
Instead of trying to convince skeptical leadership with a presentation, carve out a small part of your budget to run a real-world test of your creative idea. Present the superior results from your experiment. Data from a live campaign is far more persuasive than a theoretical argument.
Instead of only testing minor changes on a finished product, like button color, use A/B testing early in the development process. This allows you to validate broad behavioral science principles, such as social proof, for your specific challenge before committing to a full build.
To develop your "people sense," actively predict the outcomes of A/B tests and new product launches before they happen. Afterward, critically analyze why your prediction was right or wrong. This constant feedback loop on your own judgment is a tangible way to develop a strong intuition for user behavior and product-market fit.
When data from split tests is ambiguous, let your genuine enthusiasm for a particular customer segment guide your decision. This emotional investment translates into a better product and a more resilient business strategy.
