A founder demonstrated how an AI agent can watch live user sessions, analyze conversion behavior, and then autonomously create and deploy A/B tests for an app's paywall. This compresses a process that previously took months of manual work by a growth team into a single night with one prompt.

Anthropic is developing a system called "CASH" to automate growth work. It uses Claude to identify opportunities, build features (like copy changes), test them, and analyze results. The system is already delivering results comparable to a junior PM and is expected to handle increasingly complex experiments.

While AI-powered code generation gets the attention, the most significant productivity gain for engineering teams is achieving 100% automated test coverage. This is the true unlock, as it eliminates the primary bottleneck to shipping high-quality code faster, reducing bug-fixing cycles and customer support loads.

Instead of aiming for a single perfect campaign, use AI to rapidly launch a high volume of 'above-average' experiments. The ability to iterate and correct mistakes a day later makes the low cost of being wrong a strategic advantage, favoring speed over polish.

AI agents can continuously experiment with variables like subject lines, send times, and offers for each individual user. This level of granular, ongoing A/B testing is impossible to manage manually, unlocking significant performance lifts that compound over time.

Using plain-English rule files in tools like Cursor, data teams can create reusable AI agents that automate the entire A/B test write-up process. The agent can fetch data from an experimentation platform, pull context from Notion, analyze results, and generate a standardized report automatically.

Use Autoresearch to automate experimentation at a massive scale. This allows an agency to offer a compelling value proposition: running hundreds of tests for the same price as competitors who only run a few, leading to faster optimization and better results.

The common view of AI is to increase efficiency or replace headcount. A more powerful approach is to maintain your team and leverage AI for abundance. Use it to triple your output, running five marketing campaigns instead of one and exploring numerous variations to dramatically increase growth.

Instead of individual use, engineers on OpenAI's growth team created a shared, reusable Codex "skill" for the entire experiment review process. By pointing it to a Statsig experiment, the skill writes hypotheses, monitors progress, and generates a post-mortem with recommendations.