Unlike the cloud market with high switching costs, LLM workloads can be moved between providers with a single line of code. This creates insane market dynamics where millions in spend can shift overnight based on model performance or cost, posing a huge risk to the LLM providers themselves.
Related Insights
Recognizing there is no single "best" LLM, AlphaSense built a system to test and deploy various models for different tasks. This allows them to optimize for performance and even stylistic preferences, using different models for their buy-side finance clients versus their corporate users.
Unlike traditional APIs, LLMs are hard to abstract away. Users develop a preference for a specific model's 'personality' and performance (e.g., GPT-4 vs. 3.5), making it difficult for applications to swap out the underlying model without user notice and pushback.
While individual AI companies see slightly lower retention than SaaS, Stripe's data reveals customers often churn from one provider directly to a competitor, and sometimes switch back. This indicates the problem being solved is highly valued, and the churn reflects a rapidly evolving, competitive market, not a lack of product-market fit for the category itself.
The ease of building applications on top of powerful LLMs will lead companies to create their own custom software instead of buying third-party SaaS products. This shift, combined with the risk of foundation models moving up the stack, signals the end of the traditional SaaS era.
AI companies operate under the assumption that LLM prices will trend towards zero. This strategic bet means they intentionally de-prioritize heavy investment in cost optimization today, focusing instead on capturing the market and building features, confident that future, cheaper models will solve their margin problems for them.
The AI industry is not a winner-take-all market. Instead, it's a dynamic "leapfrogging" race where competitors like OpenAI, Google, and Anthropic constantly surpass each other with new models. This prevents a single monopoly and encourages specialization, with different models excelling in areas like coding or current events.
Startups are becoming wary of building on OpenAI's platform due to the significant risk of OpenAI launching competing applications (e.g., Sora for video), rendering their products obsolete. This "platform risk" is pushing developers toward neutral providers like Anthropic or open-source models to protect their businesses.
Unlike traditional SaaS where high switching costs prevent price wars, the AI market faces a unique threat. The portability of prompts and reliance on interchangeable models could enable rapid commoditization. A price war could be "terrifying" and "brutal" for the entire ecosystem, posing a significant downside risk.
Companies are becoming wary of feeding their unique data and customer queries into third-party LLMs like ChatGPT. The fear is that this trains a potential future competitor. The trend will shift towards running private, open-source models on their own cloud instances to maintain a competitive moat and ensure data privacy.
The AI value chain flows from hardware (NVIDIA) to apps, with LLM providers currently capturing most of the margin. The long-term viability of app-layer businesses depends on a competitive model layer. This competition drives down API costs, preventing model providers from having excessive pricing power and allowing apps to build sustainable businesses.
