The success of neural networks on problems like Go and protein folding, long considered intractable NP-hard problems, is profound. It suggests our formal understanding of computational hardness, which focuses on worst-case scenarios, may be an incomplete model for how to find useful, approximate solutions in practice.

Instead of 'hill climbing' on public benchmarks like Terminal Bench, Factory focuses on solving difficult software problems from enterprise customers. This creates a proprietary dataset of realistic challenges that, when solved, naturally leads to strong performance on public benchmarks as a side effect.

Go's search space is larger than the number of atoms in the universe, making exhaustive search impossible. AlphaGo's core breakthrough was using neural networks to intelligently guide its search, evaluating only the most promising moves and making an intractable problem solvable.

Top LLMs like Claude 3 and DeepSeek score 0% on complex Sudoku puzzles, a task humans can solve. This isn't a minor flaw but a categorical failure, exposing the transformer architecture's inability to handle constraint satisfaction problems that require backtracking and parallel reasoning, unlike its sequential, token-by-token processing.

Success on constraint-satisfaction puzzles like Sudoku signals a shift from current AI that summarizes existing information to a new class capable of 'generative strategy.' These models can analyze constraints and creatively propose novel solutions, tackling real-world planning problems in medicine, law, and operations rather than just describing what's already known.

The "bitter lesson" in AI research posits that methods leveraging massive computation scale better and ultimately win out over approaches that rely on human-designed domain knowledge or clever shortcuts, favoring scale over ingenuity.

Humanity's intellectual pursuits, from science to engineering, inherently focus on problems where a potential solution can be verified upon discovery. We wouldn't begin searching for something if we couldn't recognize it once found, which is the definition of an NP problem.

Don't default to AI. A simple rule-based system (heuristics) is superior when results must be fully explainable (e.g., tax software), when clear domain rules already exist, when data is limited, or when development speed is the absolute top priority.

While problems like protein folding are NP-hard in theory, the instances found in nature have structural properties that allow for efficient solutions. Real-world cases of NP-hard problems aren't the adversarial, worst-case scenarios used in complexity proofs, explaining the gap between theory and practice.