Current LLMs are intelligent enough for many tasks but fail because they lack access to complete context—emails, Slack messages, past data. The next step is building products that ingest this real-world context, making it available for the model to act upon.

A primary reason users abandon AI-driven learning is the "re-engagement barrier." After pausing on a difficult concept, they lose the immediate context. Returning requires too much cognitive effort to get back up to speed, creating a cycle of guilt and eventual abandonment that AI tools must solve for.

LLMs shine when acting as a 'knowledge extruder'—shaping well-documented, 'in-distribution' concepts into specific code. They fail when the core task is novel problem-solving where deep thinking, not code generation, is the bottleneck. In these cases, the code is the easy part.

People struggle with AI prompts because the model lacks background on their goals and progress. The solution is 'Context Engineering': creating an environment where the AI continuously accumulates user-specific information, materials, and intent, reducing the need for constant prompt tweaking.

The current limitation of LLMs is their stateless nature; they reset with each new chat. The next major advancement will be models that can learn from interactions and accumulate skills over time, evolving from a static tool into a continuously improving digital colleague.

Karpathy identifies a key missing piece for continual learning in AI: an equivalent to sleep. Humans seem to use sleep to distill the day's experiences (their "context window") into the compressed weights of the brain. LLMs lack this distillation phase, forcing them to restart from a fixed state in every new session.

General LLMs are powerful but lack the core architecture of a true learning platform. A dedicated educational tool needs built-in pedagogical methods, multimodal content, and a clear structure, which is absent in a conversational, general-purpose AI that was not built for learning at its core.

Contrary to popular belief, most learning isn't constant, active participation. It's the passive consumption of well-structured content (like a lecture or a book), punctuated by moments of active reinforcement. LLMs often demand constant active input from the user, which is an unnatural way to learn.

Unlike humans, whose poor memory forces them to generalize and find patterns, LLMs are incredibly good at memorization. Karpathy argues this is a flaw. It distracts them with recalling specific training documents instead of focusing on the underlying, generalizable algorithms of thought, hindering true understanding.