Unlike medicine or engineering where mistakes are costly and feedback is slow, programming offers a learning environment that is safe, cheap, and provides instant, precise feedback via error messages. This creates a perfect loop for AI to assist, as both the problems and the errors are highly formalized.
A beginner provides a rough sketch, and the AI recognizes it as a common, well-documented problem, then produces an elegant solution. The developer credits their own design, not realizing the AI quietly translated their vague request into a much better one, creating an inflated sense of their own ability.
AI thrives in domains with fixed, written rules and searchable histories, like programming. In ambiguous areas like organizational conflict or political negotiation, where context is unwritten and lives in people's heads, its performance plummets. Its confident output masks this unreliability, posing a danger to decision-makers.
AI's ability to code seems like advanced reasoning, but it's actually just navigating the most complete archive of human knowledge ever created. Programming's version control, documentation, and forums provide a perfectly mapped territory for AI to search, not a complex problem for it to solve through intelligence.
Contrary to human intuition, a massive, well-documented domain makes an AI's job easier, not harder. More documentation provides more 'maps' for the AI to navigate. In contrast, a simple human conflict is unsolvable for an AI because its context isn't formalized or archived, creating a void of information.