Most current AI tools for sales are general large language models with a thin layer of data on top. The real productivity leap will come from future tools where deep, domain-specific knowledge—like complex enterprise sales methodologies—is embedded from the ground up.

As benchmarks become standard, AI labs optimize models to excel at them, leading to score inflation without necessarily improving generalized intelligence. The solution isn't a single perfect test, but continuously creating new evals that measure capabilities relevant to real-world user needs.

Successful vertical AI applications serve as a critical intermediary between powerful foundation models and specific industries like healthcare or legal. Their core value lies in being a "translation and transformation layer," adapting generic AI capabilities to solve nuanced, industry-specific problems for large enterprises.

Instead of using generalist AI, LookAtMedia built a "media vertical AI model" trained on over a million journalists' writing. This focused approach yields higher quality, more authentic content with a near-zero hallucination rate (less than 0.01%), which is crucial for maintaining credibility with the media.

Contrary to fears of devaluing expertise, AI makes deep experience more critical. Seasoned professionals can better prompt, guide, and spot flaws in AI output. This "context engineering" skill, honed over years, is essential for steering AI from generic results to high-quality, strategic outcomes.

AI models lack novel context and frequently produce errors. The success of an AI-first product hinges on leveraging domain experts to build the model's "muscle," provide essential context, and constantly validate its output to ensure accuracy and value.

To manage non-deterministic AI products, Shopify created an internal tool where PMs grade AI-generated outputs. This creates a "ground truth" dataset of what "good" looks like, which is then used to fine-tune a separate LLM that acts as an automated quality judge for new features and updates.

As base model capabilities converge, the key differentiator is shifting to the "agent harness"—the infrastructure, tools, and skills built around the model. For vertical AI, this is where domain expertise is injected, creating specialized agents with custom tools that outperform generalist models.

The competitive edge in AI tools is moving beyond access to powerful LLMs. The real value now lies in creating a specialized "harness" or framework—an "Ironman suit" for the model—that enables it to perform narrow, high-value tasks with precision and industry-specific nuance.