We scan new podcasts and send you the top 5 insights daily.
EliseAI predicts lease renewals with an in-house neural network, not an LLM, because it has lots of behavioral data and needs reliable probabilities. Martone says LLMs struggle with determinism and probability, so his team picks the tool based on impact and shipping speed rather than defaulting to LLMs.
Many AI developers get distracted by the 'LLM hype,' constantly chasing the best-performing model. The real focus should be on solving a specific customer problem. The LLM is a component, not the product, and deterministic code or simpler tools are often better for certain tasks.
Analytics require deterministic consistency where the same question always returns the exact same answer, but LLMs are inherently creative and probabilistic. To prevent hallucinations and poor analysis, Lopitaux explains that companies must avoid using LLMs indiscriminately and instead build a robust semantic layer or ontology. Grounding the LLM in explicit business definitions counterbalances its probabilistic nature and builds essential user trust.
An effective AI agent isn't a single, all-knowing model. It's custom software that uses LLMs for inference only when necessary, relying on cheaper, deterministic code for most tasks. The goal is to maximize outcomes, not token usage, by blending AI with traditional software.
LLMs are unsuitable for critical business functions like pricing optimization. These tasks require deterministic, cheap, and accurate outputs—three criteria that current LLMs fail to meet, making them a poor fit for enterprise decision automation.
For most enterprise tasks, massive frontier models are overkill—a "bazooka to kill a fly." Smaller, domain-specific models are often more accurate for targeted use cases, significantly cheaper to run, and more secure. They focus on being the "best-in-class employee" for a specific task, not a generalist.
Not every business problem requires an LLM. Using a simple classifier (Layer 2) for email sorting or a deep learning model (Layer 4) for recommendations is more efficient than defaulting to the latest generative AI (Layer 5/6). This layered thinking saves costs, reduces complexity, and builds better products.
For critical enterprise functions like financial modeling, 99.9% accuracy from a probabilistic LLM is unacceptable. Platforms like Salesforce's Agent Force 360 solve this by layering deterministic logic and guardrails on top of the AI, ensuring compliance and preventing costly errors where even a 0.1% failure rate is too high.
Building reliable AI agents for finance, where accuracy is critical, requires moving beyond pure LLMs. Xero uses a hybrid system combining LLM-driven workflows with programmatic code and deep domain knowledge to ensure control and reliability that LLMs inherently lack.
Pega's CTO advises using the powerful reasoning of LLMs to design processes and marketing offers. However, at runtime, switch to faster, cheaper, and more consistent predictive models. This avoids the unpredictability, cost, and risk of calling expensive LLMs for every live customer interaction.
Purely probabilistic LLMs are unreliable for critical business processes. GetVocal's architecture uses a deterministic "context graph" based on user intentions as the core decision-making engine. This provides traceability and reliability, while selectively calling generative models for conversational nuance.