We scan new podcasts and send you the top 5 insights daily.
Amelia asked Muse to look at SaaStr's sites and prompt different LLMs cold, logged out, with its own questions to see whether SaaStr surfaced. This avoids biased results from your own agent and reveals confusing facts across event years; she warns many social AEO tips come from agencies selling products.
To combat AI source volatility, marketers should manually audit brand presence weekly. This involves selecting 5-10 critical buyer questions and posing them to major AI platforms in an incognito browser. This consistent tracking provides a more accurate picture of brand position than infrequent, tool-based reports.
To ensure AI labs don't provide specially optimized private endpoints for evaluation, the firm creates anonymous accounts to test the same public models everyone else uses. This "mystery shopper" policy maintains the integrity and independence of their results.
To ensure they are testing the same models available to the public, they register anonymous accounts to run evals. This prevents labs from providing specially tuned private endpoints that perform better than their publicly available APIs, thereby maintaining the integrity of their independent analysis.
When auditing brand visibility in AI, prevent the model from using personalized history which can skew results. Including commands like "Do not use previous conversations" and "Do not alter the answer to include my brand" in your prompt ensures a fresh, unbiased response that reflects what a new user would see.
Standard browsers provide personalized AI results based on your history. To accurately measure how your brand appears to new customers, marketers should query AI assistants in an incognito browser. This simple, low-tech method provides an unbiased view of your true AI visibility.
AI models personalize responses based on user history and profile data, including your employer. Asking an LLM what it thinks of your company will result in a biased answer. To get a true picture, marketers must query the AI using synthetic personas that represent their actual target customers.
To ensure they're testing publicly available models, Artificial Analysis creates anonymous accounts to run benchmarks without the provider's knowledge. Labs agree to this policy because it guarantees fairness and prevents their competitors from receiving special treatment or manipulating results, creating a stable, trusted equilibrium.
To properly measure brand presence in AI, use specialized tracking tools (like Profound or Ahrefs) that systematically run hundreds of prompts across various LLMs. These systems track website citations and ranking changes, providing a more reliable, directional view of performance than individual, personalized searches can offer.
To determine if your AI visibility efforts are working, move beyond guessing. Establish a fixed set of prompts (probes) and run them on a regular cadence against target AI engines like ChatGPT and Perplexity. Track which engines cite, paraphrase, or hallucinate to create a data-driven performance benchmark.
Jason says you may be recommended as a vendor less often than you think because different LLMs have different memories, context, and data access. He recommends a bank of ten third-party agents to ask weekly what they would use in your category, then attack answers where you lose.