Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Arielle, Every's Frontier Wrangler, organizes experiments so learnings can be synthesized into content, products, and consulting concepts. She also built a dashboard that tracks token spend by person and work category, making the experiment ROI visible so the team can see which experiments are worth continuing.

Related Insights

Arielle says as models improve, her ability to build systems grows. As an operator, she previously waited for vendors or engineers to add tracking features; now she can create dashboards like the token tracker herself. That shift changes what non-technical operators can own without starting an engineering project.

While heavy token spend can help you "live in the future," real value comes from communal discovery. Anthropic found that having the entire company experiment in a public Slack channel led to magical, emergent use cases as people rapidly built upon each other's ideas.

Snowflake established a cross-functional AI council with volunteers who dedicate 10-20% of their time to experimentation. This avoids chaotic, duplicated efforts from a company-wide mandate. The council then shares learnings and rolls out proven use cases to the broader team quarterly, ensuring structured adoption.

Sendbird tracks and ranks every employee's daily AI token usage on a public dashboard, categorizing them from 'AI Newbie' to 'AI God.' This gamified metric makes AI adoption a visible, shared company objective and identifies who needs enablement.

A product leader at Zapier moved beyond individual "second brains" to create a centralized "company brain." This system aggregates company strategy, design systems, and project context for AI agents to access. This created a step-change in productivity, putting a "jetpack" on every product person's back.

Using plain-English rule files in tools like Cursor, data teams can create reusable AI agents that automate the entire A/B test write-up process. The agent can fetch data from an experimentation platform, pull context from Notion, analyze results, and generate a standardized report automatically.

Dan argues companies are stuck choosing between chasing every new AI tool and shipping reliable work. A frontier team creates separation: a small group experiments and pushes only proven ideas into products, so the rest of the company can keep executing without monitoring every model release.

Kieran says a frontier team must inspire and listen to the wider organization. Running a hundred experiments without helping some stick just overwhelms people; the team's job is integration, making sure colleagues understand and adopt what works. If experiments never land, the organization cannot benefit from them.

Instead of individual use, engineers on OpenAI's growth team created a shared, reusable Codex "skill" for the entire experiment review process. By pointing it to a Statsig experiment, the skill writes hypotheses, monitors progress, and generates a post-mortem with recommendations.

A growth engineer describes automating his entire A/B testing pipeline, viewing the setup and monitoring as "boring" work. By offloading this to AI, he frees up cognitive space to focus on higher-leverage, creative problems like "what should we test next?" rather than the mechanics of the tests themselves.