Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

Adam Wood says open-weight models were a recurring theme, with a legal software company benchmarking 18 models across 200 criteria to pick the right one. If inference costs fall and software firms hold pricing, he floats a medium-term bull case that gross margins may hold better than expected.

Related Insights

Developers are increasingly using expensive frontier models only for high-stakes tasks, flocking to cheaper, open-weight alternatives for everything else. This price pressure is commoditizing AI, which could implode the debt-fueled valuations of leading AI companies.

Assuming software inference expenses will decrease does not guarantee expanding margins because declining compute costs benefit all competitors equally. AI startups demonstrating high gross margins early on offer safer evidence of proprietary defensive value, whereas low-margin companies risk remaining trapped in low-multiple structures.

AI companies operate under the assumption that LLM prices will trend towards zero. This strategic bet means they intentionally de-prioritize heavy investment in cost optimization today, focusing instead on capturing the market and building features, confident that future, cheaper models will solve their margin problems for them.

During major technology shifts like the move to cloud or AI, the best companies (e.g., hyperscalers, Snowflake) often have terrible early margins. In AI, inference costs are falling so rapidly that a company's margin profile can improve dramatically. Judging an early AI company on SaaS-era margin expectations is a mistake.

Software has long commanded premium valuations due to near-zero marginal distribution costs. AI breaks this model. The significant, variable cost of inference means expenses scale with usage, fundamentally altering software's economic profile and forcing valuations down toward those of traditional industries.

Scott compares open-weight AI to Old Navy and Southwest: roughly 80% of the frontier product at a fraction of the price. He cites UK testing where open models narrowed the cybersecurity gap, with GLM at $6 per task versus $15 for Claude Opus. That pricing gap pressures premium labs’ margins and valuations.

Despite discovering optimizations that cut inference costs by over 50%, OpenAI is expected to use these gains to improve its own gross margins ahead of a potential public offering. They will likely only pass savings to customers if competitively pressured by rivals like Anthropic, prioritizing financial health over immediate price wars.

Traditional SaaS companies enjoy roughly 95% gross margins because code is written once and deployed infinitely with negligible distribution costs. Generative AI, by contrast, incurs substantial compute overhead on every single inference, slashing margins down to 30% or 40% and breaking classic software economics.

Traditional SaaS metrics like 80%+ gross margins are misleading for AI companies. High inference costs lower margins, but if the absolute gross profit per customer is multiples higher than a SaaS equivalent, it's a superior business. The focus should shift from margin percentages to absolute gross profit dollars and multiples.

The AI market is not a 'winner-take-all' race for the single best model. Instead, developers are opting for the 'cheapest acceptable' open-weight models for most tasks. This segments the market, reserving expensive frontier models only for the most high-stakes, complex work.