Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Pitié says individually winning tests can improve conversion, but without stepping back the homepage becomes a mess, he says. Agentic experimentation can propose and prioritize tests, yet humans still approve, publish, and apply brand standards so local wins do not damage the whole customer experience.

Related Insights

Marketing decisions should not be based on internal team members' subjective preferences, such as "I wouldn't click on that." Your team is not your target audience. A culture of A/B testing ideas should always take precedence over personal opinions to avoid a bad marketing environment.

Launching experiments without prior customer interviews or market analysis is a waste of resources. The most effective experiments are designed to answer specific questions that arise from a solid research foundation, not to substitute for it.

While useful, A/B tests are often misused by management to avoid making a judgment call. If a test result is 51% vs. 49%, it's not a clear directive; it's a call for human leadership and conviction, which tests can never replace.

True speed isn't shipping broken products to everyone; it is responsible iteration with opt-in user groups. This approach distinguishes valuable A/B experiments from unacceptable "spaghetti at the wall" testing by targeting willing early adopters who understand the experimental status.

Many marketers equate CRO with just A/B testing. However, a successful program is built on two pillars: research (gathering quantitative and qualitative data) and testing (experimentation). Overlooking the research phase leads to uninformed tests and poor results, as it provides the necessary insights for what to test.

AI loops and tools like `/goal` are effective for quickly building experimental prototypes where fine details are unimportant. For building a polished product where details and unique "sauce" matter, the human-in-the-loop approach remains superior and more cost-effective.

Foster a culture of experimentation by reframing failure. A test where the hypothesis is disproven is just as valuable as a 'win' because it provides crucial user insights. The program's success should be measured by the quantity of quality tests run, not the percentage of successful hypotheses.

For a rapidly scaling brand, optimizing for small, single-digit percentage gains is a waste of time. Prioritize tests that have the potential for massive, double-digit improvements. If a test doesn't show a clear, significant winner quickly, abandon it and move to the next big idea.

When brand teams resist testing simpler, text-based emails, don't argue about aesthetics. Frame the proposal around business value: reduced design and QA time, and the potential for higher conversion rates. Quantify the impact on efficiency and revenue to get buy-in.

Traditional A/B testing is fundamentally broken for modern marketing. It's a manual process that doesn't scale, its single-variable nature prevents deep, meaningful tests, and human selection of what to test introduces significant confirmation bias, limiting true discovery and innovation.

Agentic testing needs human oversight to prevent homepage chaos | RiffOn