Over time, prompts become long and complex, accumulating contradictions from multiple contributors. Chip Huyen suggests treating them like a codebase: use another AI to analyze the prompt for inconsistencies and "refactor" it for better performance and clarity.

Goal-based loops run until an outcome is validated. If the success criteria are poorly defined, the agent will continuously burn tokens in a potentially fruitless effort. This makes precise prompt engineering and evaluation criteria critical for cost control.

Don't give LLMs full control. Use deterministic code for core logic, validation, and enforcing rules. Delegate only tasks requiring flexibility or understanding of unstructured input to the LLM, treating it as a specialized component, not the entire system.

The current ease of delegating tasks to AI with a single sentence is a temporary phenomenon. As users tackle more complex systems, the real work will involve maintaining detailed specifications and high-level architectural guides to ensure the AI agent stays on track, making prompting a more rigorous discipline.

Similar to technical debt in software, "agent debt" arises from quickly hacking together agent workflows without refinement. Over time, this leads to polluted memory, conflicting system prompts, and overlapping tools, causing the agent to behave erratically and become difficult to debug or maintain.

Prompts are effective for guiding a model's probabilistic behavior but are too unreliable for enforcing critical business logic. Use deterministic software—like schema validation and finite state machines—to guarantee rules and prevent production failures.

OpenAI found that removing repeated instructions from old prompts improved scores by 10-15% while cutting token usage by 66%. The complex rule lists built for older models now confuse systems like GPT-5.6, leading to worse and more expensive answers.

As AI models become more capable, overly detailed system prompts with many examples and hard constraints can be counterproductive. They limit the model's creativity. The Claude Code team cut their system prompt by 80% because the smarter model needs more freedom to find optimal solutions.

Teams often try to fix data extraction errors by adding complex instructions to prompts. This fails because the root cause is a structural data engineering problem, not a semantic one. The LLM receives scrambled text tokens before it can even process the prompt's instructions, making the effort futile.