The brain's hardware limitations, like slow and stochastic neurons, may actually be advantages. These properties seem perfectly suited for probabilistic inference algorithms that rely on sampling—a task that requires explicit, computationally-intensive random number generation in digital systems. Hardware and algorithm are likely co-designed.

Solving key AI weaknesses like continual learning or robust reasoning isn't just a matter of bigger models or more data. Shane Legg argues it requires fundamental algorithmic and architectural changes, such as building new processes for integrating information over time, akin to an episodic memory.

A "software-only singularity," where AI recursively improves itself, is unlikely. Progress is fundamentally tied to large-scale, costly physical experiments (i.e., compute). The massive spending on experimental compute over pure researcher salaries indicates that physical experimentation, not just algorithms, remains the primary driver of breakthroughs.

Musk highlights that the human brain built civilization using just 10 watts for higher functions. This serves as a clear benchmark, demonstrating that current AI supercomputers, which consume megawatts, have a massive, untapped opportunity for improving power efficiency.

AI's capabilities are highly uneven. Models are already superhuman in specific domains like speaking 150 languages or possessing encyclopedic knowledge. However, they still fail at tasks typical humans find easy, such as continual learning or nuanced visual reasoning like understanding perspective in a photo.

The "bitter lesson" in AI research posits that methods leveraging massive computation scale better and ultimately win out over approaches that rely on human-designed domain knowledge or clever shortcuts, favoring scale over ingenuity.

Instead of seizing human industry, a superintelligent AI could leverage its understanding of biology to create its own self-replicating systems. It could design organisms to grow its computational hardware, a far faster and more efficient path to power than industrial takeover.

The next leap in AI will come from integrating general-purpose reasoning models with specialized models for domains like biology or robotics. This fusion, creating a "single unified intelligence" across modalities, is the base case for achieving superintelligence.

Shane Legg, a pioneer in the field, maintains his original 2009 prediction that there is a 50/50 probability of achieving "minimal AGI" by 2028. He defines this as an AI agent capable of performing the cognitive tasks of a typical human.