Instead of relying solely on 'black box' LLMs, a more robust approach is neurosymbolic computation. This method combines three estimators: a traditional symbolic/rule-based model (e.g., a medical checklist), a neural network prediction, and an LLM's assessment. By comparing these diverse outputs, experts can make more informed and reliable judgments.
Related Insights
Simply creating an LLM judge prompt isn't enough. Before deploying it, you must test its alignment with human judgment. Run the judge on your manually labeled data and analyze the results in a confusion matrix. This helps you see where it disagrees with you (false positives/negatives) so you can refine the prompt and build trust.
LLMs predict the next token in a sequence. The brain's cortex may function as a general prediction engine capable of "omnidirectional inference"—predicting any missing information from any available subset of inputs, not just what comes next. This offers a more flexible and powerful form of reasoning.
A practical hack to improve AI agent reliability is to avoid built-in tool-calling functions. LLMs have more training data on writing code than on specific tool-use APIs. Prompting the agent to write and execute the code that calls a tool leverages its core strength and produces better outcomes.
An LLM shouldn't do math internally any more than a human would. The most intelligent AI systems will be those that know when to call specialized, reliable tools—like a Python interpreter or a search API—instead of attempting to internalize every capability from first principles.
AI expert Andrej Karpathy suggests treating LLMs as simulators, not entities. Instead of asking, "What do you think?", ask, "What would a group of [relevant experts] say?". This elicits a wider range of simulated perspectives and avoids the biases inherent in forcing the LLM to adopt a single, artificial persona.
To ensure reliability in healthcare, ZocDoc doesn't give LLMs free rein. It wraps them in a hybrid system where traditional, deterministic code orchestrates the AI's tasks, sets firm boundaries, and knows when to hand off to a human, preventing the 'praying for the best' approach common with direct LLM use.
AI and formal methods have been separate fields with opposing traits: AI is flexible but untrustworthy, while formal methods offer guarantees but are rigid. The next frontier is combining them into neurosymbolic systems, creating a "peanut butter and chocolate" moment that captures the best of both worlds.
Arvind Krishna firmly believes that today's LLM technology path is insufficient for reaching Artificial General Intelligence (AGI). He gives it extremely low odds, stating that a breakthrough will require fusing current models with structured, hard knowledge, a field known as neurosymbolic AI, before AGI becomes plausible.
Instead of treating a complex AI system like an LLM as a single black box, build it in a componentized way by separating functions like retrieval, analysis, and output. This allows for isolated testing of each part, limiting the surface area for bias and simplifying debugging.
To improve LLM reasoning, researchers feed them data that inherently contains structured logic. Training on computer code was an early breakthrough, as it teaches patterns of reasoning far beyond coding itself. Textbooks are another key source for building smaller, effective models.
