Resist the urge to apply LLMs to every problem. A better approach is using a 'first principles' decision tree. Evaluate if the task can be solved more simply with data visualization or traditional machine learning before defaulting to a complex, probabilistic, and often overkill GenAI solution.
Related Insights
Many AI developers get distracted by the 'LLM hype,' constantly chasing the best-performing model. The real focus should be on solving a specific customer problem. The LLM is a component, not the product, and deterministic code or simpler tools are often better for certain tasks.
LLMs shine when acting as a 'knowledge extruder'—shaping well-documented, 'in-distribution' concepts into specific code. They fail when the core task is novel problem-solving where deep thinking, not code generation, is the bottleneck. In these cases, the code is the easy part.
An LLM shouldn't do math internally any more than a human would. The most intelligent AI systems will be those that know when to call specialized, reliable tools—like a Python interpreter or a search API—instead of attempting to internalize every capability from first principles.
High productivity isn't about using AI for everything. It's a disciplined workflow: breaking a task into sub-problems, using an LLM for high-leverage parts like scaffolding and tests, and reserving human focus for the core implementation. This avoids the sunk cost of forcing AI on unsuitable tasks.
The goal of testing multiple AI models isn't to crown a universal winner, but to build your own subjective "rule of thumb" for which model works best for the specific tasks you frequently perform. This personal topography is more valuable than any generic benchmark.
Conceptualize Large Language Models as capable interns. They excel at tasks that can be explained in 10-20 seconds but lack the context and planning ability for complex projects. The key constraint is whether you can clearly articulate the request to yourself and then to the machine.
A 'GenAI solves everything' mindset is flawed. High-latency models are unsuitable for real-time operational needs, like optimizing a warehouse worker's scanning path, which requires millisecond responses. The key is to apply the right tool—be it an optimizer, machine learning, or GenAI—to the specific business problem.
Don't assume AI can effectively perform a task that doesn't already have a well-defined standard operating procedure (SOP). The best use of AI is to infuse efficiency into individual steps of an existing, successful manual process, rather than expecting it to complete the entire process on its own.
Pega's CTO advises using the powerful reasoning of LLMs to design processes and marketing offers. However, at runtime, switch to faster, cheaper, and more consistent predictive models. This avoids the unpredictability, cost, and risk of calling expensive LLMs for every live customer interaction.
Instead of being swayed by new AI tools, business owners should first analyze their own processes to find inefficiencies. This allows them to select a specific tool that solves a real problem, thereby avoiding added complexity and ensuring a genuine return on investment.
