Overly structured, workflow-based systems that work with today's models will become bottlenecks tomorrow. Engineers must be prepared to shed abstractions and rebuild simpler, more general systems to capture the gains from exponentially improving models.

Early agent harnesses were rigid scaffolds designed to force models along a specific path. As models become more intelligent and steerable, much of this scaffolding is no longer needed and can be deleted. The focus of modern harnesses is now on enabling longer, more complex execution chains.

While building intricate frameworks (scaffolding) to correct model behavior is effective now, it may become obsolete. The speaker suggests it's better to focus on giving models more fundamental capabilities and trust that future, more generalized models will handle tasks without needing such hand-holding.

Overly prescriptive fixes or guardrails for current AI weaknesses are temporary. As models improve, this "scaffolding" becomes obsolete. Teams must stay flexible and be ready to remove old constraints with each new model release, rather than over-engineering for today's problems.

The "bitter lesson" of AI applies to product development: complex scaffolding built around model limitations (like early vector stores or agent frameworks) will inevitably become obsolete as the models themselves get smarter and absorb those functions. Don't over-engineer solutions that a future model will solve natively.

Building on AI involves a "tick-tock" cycle. First, engineers create a complex "harness" of prompts and skills. Then, a new, more powerful base model is released that performs those skills natively, "eating the harness" and forcing engineers to simplify and build a new layer of more advanced heuristics.

When building with rapidly evolving LLMs, avoid creating rigid structures or "scaffolding" to compensate for current model weaknesses. This technical debt becomes a liability when more capable models emerge. Instead, design systems that can leverage future improvements without a complete rebuild.

Building on AI requires creating custom infrastructure to fill performance gaps. As underlying models improve, founders must be prepared to delete this now-redundant code and upgrade their product vision to tackle the next set of challenges at the new frontier. This cycle of building and deleting is key to staying innovative.

The development path for AI models follows a pattern: a new capability (e.g., better prompting, multi-shot editing) is first implemented as a separate "harness" or scaffold around the core model. Over time, this external logic is absorbed directly into the model's architecture and learned end-to-end.