The choice of optimization algorithm dictates which model architectures are viable. While weaker optimizers require simpler models, a more powerful optimizer can successfully train more complex, harder-to-optimize architectures. This shows how optimizer and architecture research are deeply intertwined, unlocking new performance possibilities.

The NEX vs. EX problem is equivalent to solving highly compressed, structured SAT instances. Because real-world SAT instances are also highly structured (not random), the possibility that NEX=EX implies that structure might be the key to efficiently solving certain exponential-time problems.

Liquid AI's early, highly effective non-linear models faced a major scaling bottleneck. Non-linear relationships are difficult to "tensorize"—convert from sequential to parallel computations—which is essential for GPU efficiency. This is why linear systems like state-space models scale more easily.

When a mathematical optimization model is "infeasible," it's not a failure. It's a valuable diagnostic tool that proves a set of business rules, budgets, or constraints are fundamentally in conflict and cannot all be satisfied simultaneously. This forces clarification of priorities.

Use LLMs to help define business problems, write code, and identify potential constraints. Then, hand off to a mathematical solver like Gurobi, which provides a mathematically guaranteed optimal solution that an LLM cannot, as it will never violate a hard constraint.

A major breakthrough for Liquid AI was finding a closed-form solution for the differential equations governing their neural networks, a problem unsolved since 1907. This eliminated the need for slow, step-by-step numerical solvers, enabling a massive leap in scalability from hundreds to potentially billions of neurons.

Third-party tracker METR observed that model complexity was doubling every seven months. However, a recent proprietary model shattered this trend, demonstrating nearly double the expected capability for independent operation (15 hours vs. an expected 8). This signals that AI advancement is accelerating unpredictably, outpacing prior scaling laws.

The market often misinterprets AI progress as linear. However, a clear 'scaling law' dictates that a tenfold increase in the computing power used to train LLMs results in a twofold capability improvement. This exponential relationship means future advancements will be far more disruptive and surprising than incremental projections suggest.

When a model is infeasible due to conflicting constraints, Gurobi's "FeasRelax" function calculates the minimum adjustment needed to find a solution. This provides a precise answer like, "If we increase the budget by 10%, the plan becomes feasible."