Instead of training models to generalize across many problems, this approach focuses on finding the single best solution for one specific task, like a new material or algorithm. The model itself can be discarded; the value is in the single, world-changing artifact it produces.

LoRa training focuses computational resources on a small set of additional parameters instead of retraining the entire 6B parameter z-image model. This cost-effective approach allows smaller businesses and individual creators to develop highly specialized AI models without needing massive infrastructure.

The model uses a Mixture-of-Experts (MoE) architecture with over 200 billion parameters, but only activates a "sparse" 10 billion for any given task. This design provides the knowledge base of a massive model while keeping inference speed and cost comparable to much smaller models.

Low-Rank Adaptation (LoRa) allows a single base AI model to be efficiently fine-tuned into multiple, distinct specialist models. This is a powerful strategy for companies needing varied editing capabilities, such as for different client aesthetics, without the high cost of training and maintaining separate large models.

A fundamental constraint today is that the model architecture used for training must be the same as the one used for inference. Future breakthroughs could come from lifting this constraint. This would allow for specialized models: one optimized for compute-intensive training and another for memory-intensive serving.

After two decades of experience and carefully tuning a model by hand, Karpathy was surprised when his automated research agent, running overnight, discovered superior hyperparameter configurations he had missed. This shows AI's power to surpass deep human expertise in objective optimization tasks.

Specialized AI models no longer require massive datasets or computational resources. Using LoRA adaptations on models like FLUX.2, developers and creatives can fine-tune a model for a specific artistic style or domain with a small set of 50 to 100 images, making custom AI accessible even with limited hardware.

Instead of only analyzing a fully trained model, "intentional design" seeks to control what a model learns during training. The goal is to shape the loss landscape to produce desired behaviors and generalizations from the outset, moving from archaeology to architecture.

Instead of running hundreds of brute-force experiments, machine learning models analyze historical data to predict which parameter combinations will succeed. This allows teams to focus on a few dozen targeted experiments to achieve the same process confidence, compressing months of work into weeks.