A technique from cryptography, the Feistel network, makes any function invertible. When applied to neural network layers ("RevNets"), it allows activations from the forward pass to be re-calculated during the backward pass instead of stored. This trades extra compute for a massive reduction in memory footprint during training.

Simulating strategies with memory (like "grim trigger") or with multiple players causes an exponential explosion of simulation branches. This can be solved by having all simulated agents draw from the same shared sequence of random numbers, which forces all simulation branches to halt at the same conceptual "time step."

Breakthroughs like neural network "pruning" can reduce model size by 90% without losing accuracy, offering a 10x reduction in inference costs. This highlights that algorithmic innovation, not just acquiring more hardware, will be a key competitive vector in the AI race, enabling more output with less energy.

MIT Professor Ryan Williams operates as if the Strong Exponential Time Hypothesis (SETH) is false. This belief forces him to discard standard approaches and explore novel algorithmic ideas. His failed attempts to refute SETH have led to unexpected solutions for other important problems.

It's possible to solve problems like finding the majority element in a bit string using constant memory, regardless of the string's length. This is achieved by encoding computations as sequences of operations in a non-commutative group, defying the intuition that counting requires logarithmic space to store a counter.

Professor Williams assigns only 80% confidence to P != NP, lower than his peers. His rationale is that our intuition about computational limits is frequently proven wrong by surprising new algorithms. The vast, unexplored space of algorithms makes a definitive conclusion more uncertain than widely believed.

EnCharge AI's innovation was to reframe in-memory analog compute not as a scaled-up memory problem, but as a high-precision analog design problem. They borrowed techniques from medical and aerospace circuits to overcome noise and enable massive efficiency gains.

Current AI models become exponentially more expensive as input size grows (quadratic scaling). New "subquadratic" architectures, however, scale linearly by pre-selecting relevant data. This change could slash compute costs by orders of magnitude, making massive context windows economically viable.

A core legacy of AlphaGo is turning complex search problems into 'games' for AI agents. AlphaTensor reframed the challenge of finding the fastest matrix multiplication algorithm as a game, allowing it to discover a more efficient method than any human had found in over 50 years, proving the approach's power for scientific discovery.