Companies with valuable proprietary data should not license it away. A better strategy to guide foundation model development is to keep the data private but release public benchmarks and evaluations based on it. This incentivizes LLM providers to train their models on the specific tasks you care about, improving their performance for your product.

The company provides public benchmarks for free to build trust. It monetizes by selling private benchmarking services and subscription-based enterprise reports, ensuring AI labs cannot pay for better public scores and thus maintaining objectivity.

To ensure AI labs don't provide specially optimized private endpoints for evaluation, the firm creates anonymous accounts to test the same public models everyone else uses. This "mystery shopper" policy maintains the integrity and independence of their results.

LM Arena, known for its public AI model rankings, generates revenue by selling custom, private evaluation services to the same AI companies it ranks. This data helps labs improve their models before public release, but raises concerns about a "pay-to-play" dynamic that could influence public leaderboard performance.

To maintain independence and trust, their public benchmarks are free and cannot be influenced by payments. The company generates revenue by selling detailed reports and insight subscriptions to enterprises, and by conducting private, custom benchmarking for AI companies, separating their public good from their commercial offerings.

LM Arena's $1.7B valuation stems from its innovative flywheel: it attracts millions of users to a simple "pick your favorite AI" game, generating data that becomes the industry's most trusted leaderboard. This forces major AI labs to pay for evaluations, turning a user engagement loop into a powerful marketing and revenue engine.

Instead of gating its valuable review data like traditional analyst firms, G2 strategically chose to syndicate it and make it available to LLMs. This ensures G2 remains a trusted, cited source within AI-generated answers, maintaining brand influence and relevance where buyers are now making decisions.

Stack Overflow structures its AI data licensing deals as recurring revenue streams, not one-time payments. AI labs pay for ongoing rights to train new models on the entire cumulative dataset, ensuring the corpus's value is monetized continuously as the AI industry evolves.

Perplexity achieves profitability on its paid subscribers, countering the narrative of unsustainable AI compute costs. Critically, the cost of servicing free users is categorized as a research and development expense, as their queries are used to train and improve the system. This accounting strategy presents a clearer path to sustainable unit economics for AI services.