Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

When introducing experimentation, the most significant outcome isn't the initial metric lift but the cultural shift. Getting a department to run its first-ever randomized trial builds the technical apparatus and cultural buy-in for future, more ambitious testing, a victory worth celebrating regardless of the result.

Related Insights

Marketing decisions should not be based on internal team members' subjective preferences, such as "I wouldn't click on that." Your team is not your target audience. A culture of A/B testing ideas should always take precedence over personal opinions to avoid a bad marketing environment.

Foster a culture of experimentation by reframing failure. A test where the hypothesis is disproven is just as valuable as a 'win' because it provides crucial user insights. The program's success should be measured by the quantity of quality tests run, not the percentage of successful hypotheses.

Beyond testing hypotheses, real-world experiments serve a crucial social function: reducing employee fear of change. By co-designing experiments with skeptics to test their specific assumptions, innovation teams can quell fears with data, turning organizational resistance into buy-in.

Shifting the conversation from "moving faster" to "investing wisely" helps get stakeholder buy-in. It highlights that experiments prevent wasting significant time and money on suboptimal or failing ideas, making it a powerful risk management tool.

To shift a sales-led culture, don't just present data. Let executives vote on their preferred feature version, then run an A/B test. Showing them hard data that their gut feeling was wrong is a powerful way to prove the value of a data-driven product process and secure buy-in for change.

Instead of trying to convince skeptical leadership with a presentation, carve out a small part of your budget to run a real-world test of your creative idea. Present the superior results from your experiment. Data from a live campaign is far more persuasive than a theoretical argument.

Chess.com's goal of 1,000 experiments isn't about the number. It’s a forcing function to expose systemic blockers and drive conversations about what's truly needed to increase velocity, like no-code tools and empowering non-product teams to test ideas.

Instead of only testing minor changes on a finished product, like button color, use A/B testing early in the development process. This allows you to validate broad behavioral science principles, such as social proof, for your specific challenge before committing to a full build.

To get company-wide buy-in for CRO, focus reporting on program-level metrics, not just individual test results. Share high-level insights like win/loss rates and cross-departmental impact in quarterly reviews. This frames CRO as a strategic business function, not just a series of tactical marketing experiments.

To foster innovation, leaders must give teams the freedom to experiment without fear of reprisal for failure. If every new idea is immediately judged by its short-term ROI, people will cease to try anything new. Psychological safety to test and fail is the prerequisite for a dynamic, evolving culture.

An Organization's First A/B Test Is a Cultural Victory, Not Just a Technical One | RiffOn