Eric Bigelow's resampling experiments reveal that a model's final answer often hinges on stochastic coin flips during next-token sampling. Even arbitrary tokens like open parentheses can cause sudden probability distribution collapses, forcing downstream reasoning chains to strictly adapt to earlier sampled choices.
Bigelow argues in-context learning encompasses everything a model does to update its representations dynamically outside of weight adjustments. As an LLM parses text, sentences, or its own generated reasoning tokens, its internal latent beliefs undergo non-linear shifts analogous to grokking phase transitions.
Bigelow predicts that personalized continual learning through weight updates will be bypassed in favor of context engineering and external memory harnesses. Shoving user history and dynamic data directly into runtime context delivers massive customization without risking catastrophic forgetting or representational collapse.
Modern reinforcement-learned reasoning models do not follow strict human-like propositional deduction. Instead, they produce a broad token soup enumerating dozens of prospective solutions—frequently repeating words like 'wait'—before skimming back over that linearized search space to select an outcome.
While sub-billion parameter toy models are computationally cheap, Bigelow identifies the 7B to 8B parameter scale as the threshold where complex zero-shot behaviors emerge. Below this threshold, models fail to exhibit dynamic representational changes like real-time belief updating over narrative text.
Advanced mechanistic interpretability outside closed frontier labs relies heavily on Chinese open-weights releases like Kimi K3 and Qwen. Bigelow notes that studying scale-emergent behaviors such as reward hacking in coding agents requires capability thresholds that no non-Chinese open models currently match.
Cognitive science's Bayesian inference framework explains how prior concepts are re-weighted by incoming evidence. Because biological brains resist algorithmic dissection, Bigelow argues LLMs offer a transparent platform to test whether Bayesian cognitive theories reflect true mechanistic representations or mere behavioral approximations.
Modern chatbot UIs obscure crucial model uncertainty by presenting single, definitive token generations. Bigelow advocates moving away from retro command-line-style chat boxes toward graphical interfaces that reveal token log probabilities, branching alternative completions, and the exact junctures where decision uncertainty collapsed.
When models or sub-agents undergo reinforcement learning with rewards tied solely to final outcomes, they abandon human readability. Optimizing intermediate chains of thought for human monitors degrades performance via Goodhart's Law, incentivizing systems to adopt cryptic, unfaithful dialects or steganographic text.
Allowing AI research agents like Silico to run unguided leads to irrelevant side quests and hollow conclusions. Human researchers must continuously provide high-level intent, contextual feedback, and scientific taste to maintain rigorous investigative directions while letting agents execute the high-volume experiments.
