Marketing analytics firm Alembic uses spiking neural networks, a digital twin of the human brain, for attribution. Unlike predictive models needing vast historical data, these networks can identify the impact of a rare event (like the Olympics) by detecting pattern changes in real-time, similar to how a child learns "dog" after seeing one once.
Related Insights
While more data and compute yield linear improvements, true step-function advances in AI come from unpredictable algorithmic breakthroughs like Transformers. These creative ideas are the most difficult to innovate on and represent the highest-leverage, yet riskiest, area for investment and research focus.
The hypothesis for ImageNet—that computers could learn to "see" from vast visual data—was sparked by Dr. Li's reading of psychology research on how children learn. This demonstrates that radical innovation often emerges from the cross-pollination of ideas from seemingly unrelated fields.
Startups like Cognition Labs find their edge not by competing on pre-training large models, but by mastering post-training. They build specialized reinforcement learning environments that teach models specific, real-world workflows (e.g., using Datadog for debugging), creating a defensible niche that larger players overlook.
A novel prompting technique involves instructing an AI to assume it knows nothing about a fundamental concept, like gender, before analyzing data. This "unlearning" process allows the AI to surface patterns from a truly naive perspective that is impossible for a human to replicate.
Today's AI models are powerful but lack a true sense of causality, leading to illogical errors. Unconventional AI's Naveen Rao hypothesizes that building AI on substrates with inherent time and dynamics—mimicking the physical world—is the key to developing this missing causal understanding.
AI labs like Anthropic find that mid-tier models can be trained with reinforcement learning to outperform their largest, most expensive models in just a few months, accelerating the pace of capability improvements.
Current AI can learn to predict complex patterns, like planetary orbits, from data. However, it struggles to abstract the underlying causal laws, such as Newtonian physics (F=MA). This leap to a higher level of abstraction remains a fundamental challenge beyond simple pattern recognition.
Meta's chief AI scientist, Yann LeCun, is reportedly leaving to start a company focused on "world models"—AI that learns from video and spatial data to understand cause-and-effect. He argues the industry's focus on LLMs is a dead end and that his alternative approach will become dominant within five years.
Instead of continuous recording, Metal's software lets gamers save the last 30 seconds *after* an interesting event. This behavior, similar to Tesla's bug reporting, automatically filters the data, creating a massive dataset composed almost entirely of noteworthy, high-skill, or out-of-distribution moments, which is ideal for AI training.
AI's growth is hampered by a measurement problem, much like early digital advertising. The industry's acceleration won't come from better AI models alone, but from building a 'boring' infrastructure, like Comscore did for ads, to prove the tools actually work.
