Recognizing there is no single "best" LLM, AlphaSense built a system to test and deploy various models for different tasks. This allows them to optimize for performance and even stylistic preferences, using different models for their buy-side finance clients versus their corporate users.

Effective enterprise AI deployment involves running human and AI workflows in parallel. When the AI fails, it generates a data point for fine-tuning. When the human fails, it becomes a training moment for the employee. This "tandem system" creates a continuous feedback loop for both the model and the workforce.

For specialized, high-stakes tasks like insurance underwriting, enterprises will favor smaller, on-prem models fine-tuned on proprietary data. These models can be faster, more accurate, and more secure than general-purpose frontier models, creating a lasting market for custom AI solutions.

Training AI agents to execute multi-step business workflows demands a new data paradigm. Companies create reinforcement learning (RL) environments—mini world models of business processes—where agents learn by attempting tasks, a more advanced method than simple prompt-completion training (SFT/RLHF).

AI labs like Anthropic find that mid-tier models can be trained with reinforcement learning to outperform their largest, most expensive models in just a few months, accelerating the pace of capability improvements.

The true enterprise value of AI lies not in consuming third-party models, but in building internal capabilities to diffuse intelligence throughout the organization. This means creating proprietary "AI factories" rather than just using external tools and admiring others' success.

Unlike deterministic SaaS software that works consistently, AI is probabilistic and doesn't work perfectly out of the box. Achieving 'human-grade' performance (e.g., 99.9% reliability) requires continuous tuning and expert guidance, countering the hype that AI is an immediate, hands-off solution.

While prompt optimization is theoretically appealing, OpenPipe's team evaluated methods like JEPA and found they provided only minor boosts. Their RL fine-tuning methods delivered vastly superior results (96% vs 56% on a benchmark), suggesting weight updates still trump prompt engineering for complex tasks.

Fine-tuning an AI model is most effective when you use high-signal data. The best source for this is the set of difficult examples where your system consistently fails. The processes of error analysis and evaluation naturally curate this valuable dataset, making fine-tuning a logical and powerful next step after prompt engineering.