For millions of vectors, exact search (like a FAISS flat index) is too slow. Production systems use Approximate Nearest Neighbor (ANN) algorithms which trade a small amount of accuracy for orders-of-magnitude faster search performance, making large-scale applications feasible.

The primary driver of success in large-scale model training is the ability to conduct numerous experiments daily. A robust infrastructure that minimizes cycle time for testing hypotheses provides a greater advantage than focusing solely on developing new algorithms.

Managed vector databases are convenient, but building a search engine from scratch using a library like FAISS provides a deeper understanding of index types, latency tuning, and memory trade-offs, which is crucial for optimizing AI systems.

AI's hunger for context is making search a critical but expensive component. As illustrated by Turbo Puffer's origin, a single recommendation feature using vector embeddings can cost tens of thousands per month, forcing companies to find cheaper solutions to make AI features economically viable at scale.

Teams often default to convenient embedding models from their cloud provider, treating them as interchangeable. This is a critical mistake. The choice of embedding model directly impacts retrieval quality, with specialized models offering double-digit performance gains that can unlock new capabilities.

Poolside rejected the standard practice of packaging and copying datasets to training clusters. Instead, they stream data just-in-time. This allows training to start immediately, even while data is still materializing, and makes experimenting with data mixes a simple configuration change, dramatically accelerating the research cycle.

Embeddings from different sizes of Voyage AI models are compatible. This lets teams embed production data with a large model while developers use a free, local “Nano” model for queries. This novel approach eliminates token costs during development and testing, improving developer experience.

A typical A/B test re-ranks the same set of results. However, changing the embedding model alters the fundamental retrieval step, meaning the two versions return entirely different sets of documents for the same query. This complicates analysis, as performance differences reflect both model quality and the content of the newly retrieved documents.

Criteo's models moved from using manually crafted, extremely high-dimensional sparse vectors (e.g., 2^12 features) with linear models to dense vectors (a few hundred features) automatically computed by deep learning algorithms. This shift eliminated manual feature engineering and improved model adaptability.