Making an API usable for an LLM is a novel design challenge, analogous to creating an ergonomic SDK for a human developer. It's not just about technical implementation; it requires a deep understanding of how the model "thinks," which is a difficult new research area.

Related Insights

To avoid overwhelming an LLM's context with hundreds of tools, a dynamic MCP approach offers just three: one to list available API endpoints, one to get details on a specific endpoint, and one to execute it. This scales well but increases latency and complexity due to the multiple turns required for a single action.

Simply offering the latest model is no longer a competitive advantage. True value is created in the system built around the model—the system prompts, tools, and overall scaffolding. This 'harness' is what optimizes a model's performance for specific tasks and delivers a superior user experience.

When building for AI-powered environments, design tools to be equally usable by humans and the AI model. An elegant, simple design for humans often translates directly into an effective tool for AI agents, simplifying development and promoting shared logic.

The early focus on crafting the perfect prompt is obsolete. Sophisticated AI interaction is now about 'context engineering': architecting the entire environment by providing models with the right tools, data, and retrieval mechanisms to guide their reasoning process effectively.

Open-ended prompts overwhelm new users who don't know what's possible. A better approach is to productize AI into specific features. Use familiar UI like sliders and dropdowns to gather user intent, which then constructs a complex prompt behind the scenes, making powerful AI accessible without requiring prompt engineering skills.

AI development has evolved to where models can be directed using human-like language. Instead of complex prompt engineering or fine-tuning, developers can provide instructions, documentation, and context in plain English to guide the AI's behavior, democratizing access to sophisticated outcomes.

Developing LLM applications requires solving for three infinite variables: how information is represented, which tools the model can access, and the prompt itself. This makes the process less like engineering and more like an art, where intuition guides you to a local maxima rather than a single optimal solution.

Designers need to get into code faster not just for prototyping, but because the AI model is an active participant in the user experience. You cannot fully design the user's interaction without directly understanding how this non-human "third party" behaves, responds, and affects the outcome.

Exposing a full API via the Model Context Protocol (MCP) overwhelms an LLM's context window and reasoning. This forces developers to abandon exposing their entire service and instead manually craft a few highly specific tools, limiting the AI's capabilities and defeating the "do anything" vision of agents.

Instead of forcing AI to be as deterministic as traditional code, we should embrace its "squishy" nature. Humans have deep-seated biological and social models for dealing with unpredictable, human-like agents, making these systems more intuitive to interact with than rigid software.