Faced with rising costs from proprietary labs, sophisticated enterprise clients are building internal evaluation and routing systems. This allows them to use cheaper, open-source models for less complex tasks, optimizing for both cost and performance.

Ramp created an internal AI tool that acts as a wrapper around an LLM. It's connected to Notion, Slack, and Snowflake, building a persistent memory of team activities and individual work styles. This "company brain" can diagnose business issues, summarize communications, and draft meeting prep in minutes, not weeks.

For most startups, training a custom foundation model is a waste of capital. The winning strategy is to focus on workflow and proprietary data, building a "headless" product that uses a model router to switch between the cheapest, most effective LLMs for any given task.

Fintech company Ramp is expanding into AI infrastructure by launching a 'model router.' This tool addresses growing CFO frustration with uncontrolled AI spending by intelligently routing tasks to the most cost-effective model. This move indicates that AI cost management is becoming a critical new product category for enterprise software.

To optimize AI costs and sustainability, UBS employs a "model garden" with various frontier and smaller models. An internal AI system then routes employee questions to the most appropriate, cost-effective model, preventing the wasteful use of powerful, expensive LLMs for simple, non-frontier problems.

RAMP built its AI platform in-house because they view internal productivity as a competitive moat. Owning the tool allows them to move faster, deeply understand user pain points, and leverage internal learnings to inform their external customer-facing products.

Instead of relying on a single large AI model, companies are adopting "model orchestration" to control costs. This involves using a router to send prompts to the most appropriate model based on the task, often cascading from cheap, small models to more expensive ones only when necessary.

Companies are building intelligent systems that analyze a user's prompt and automatically route it to the most cost-effective model that can handle the task. This avoids using expensive frontier models for simple requests, with some companies like Coinbase successfully keeping costs flat despite exponential usage growth.

The recent focus on model routers signals a maturation of enterprise AI strategy. The initial "growth at all costs" phase, which encouraged rampant employee use ("token maxing"), is giving way to a new era of cost optimization and demonstrating clear ROI on AI investments.