Contrary to intuition, providing AI with excessive or irrelevant information confuses it and diminishes the quality of its output. This phenomenon, called 'context rot,' means users must provide clean, concise, and highly relevant data to get the best results, rather than simply dumping everything in.
Related Insights
AI models like Claude Code can experience a decline in output quality as their context window fills. It is recommended to start a new session once the context usage exceeds 50% to avoid this degradation, which can manifest as the model 'forgetting' earlier instructions.
When using AI for complex analysis like a medical case, providing a detailed, unabridged history is crucial. The host found that when he summarized his son's case history to start a new chat, the model's performance noticeably worsened because it lacked the fine-grained, day-to-day data points for accurate trend analysis.
AI data agents can misinterpret results from large tables due to context window limits. The solution is twofold: instruct the AI to use query limits (e.g., `LIMIT 1000`), and crucially, remind it in subsequent prompts that the data it is analyzing is only a sample, not the complete dataset.
Even models with million-token context windows suffer from "context rot" when overloaded with information. Performance degrades as the model struggles to find the signal in the noise. Effective context engineering requires precision, packing the window with only the exact data needed.
The effectiveness of an AI system isn't solely dependent on the model's sophistication. It's a collaboration between high-quality training data, the model itself, and the contextual understanding of how to apply both to solve a real-world problem. Neglecting data or context leads to poor outcomes.
Long-running AI agent conversations degrade in quality as the context window fills. The best engineers combat this with "intentional compaction": they direct the agent to summarize its progress into a clean markdown file, then start a fresh session using that summary as the new, clean input. This is like rebooting the agent's short-term memory.
A critical learning at LinkedIn was that pointing an AI at an entire company drive for context results in poor performance and hallucinations. The team had to manually curate "golden examples" and specific knowledge bases to train agents effectively, as the AI couldn't discern quality on its own.
Even with large advertised context windows, LLMs show performance degradation and strange behaviors when overloaded. Described as "context anxiety," they may prematurely give up on complex tasks, claim imaginary time constraints, or oversimplify the problem, highlighting the gap between advertised and effective context sizes.
Hunt's team at Perscient found that AI "hallucinates" when given freedom. Success comes from "context engineering"—controlling all inputs, defining the analytical framework, and telling it how to think. You must treat AI like a constrained operating system, not a creative partner.
AI has no memory between tasks. Effective users create a comprehensive "context library" about their business. Before each task, they "onboard" the AI by feeding it this library, giving it years of business knowledge in seconds to produce superior, context-aware results instead of generic outputs.
