Get your free personalized podcast brief

We scan new podcasts and send you the top 5 insights daily.

To accurately measure its false positive rate (how often it wrongly flags human writing as AI), Pangram runs its models on millions of documents written before 2022. Since this content is guaranteed to be AI-free and is not in the training set, it provides a clean benchmark for accuracy.

Related Insights

Effective AI detection frames the problem as large-scale authorship identification. By training on paired examples of human versus LLM-generated text on the same prompts, detection models learn to recognize the unique statistical "smell" or stylistic signature of each major AI model.

AI detection can identify text from new LLMs because most models share a common "ancestry." They are either trained on the same foundational corpora, like Common Crawl, or fine-tuned with synthetic data from major models, giving them a detectable shared statistical fingerprint.

According to Pangram's CEO, different LLMs have unique "voices." Anthropic's Claude tends to be verbose and hedges statements, while OpenAI's ChatGPT is more curt and favors short, staccato sentences. These stylistic fingerprints allow detection tools to not only identify AI text but also its likely source.

Pangram Labs' detector isn't hard-coded. It's a deep learning model trained on millions of examples. For each human text (e.g., a Yelp review), it sees an AI-generated equivalent, learning the subtle, often inarticulable, differences in word choice and structure that separate them.

For an AI detection tool, a low false-positive rate is more critical than a high detection rate. Pangram claims a 1-in-10,000 false positive rate, which is its key differentiator. This builds trust and avoids the fatal flaw of competitors: incorrectly flagging human work as AI-generated, which undermines the product's credibility.

To distinguish between light AI assistance (like Grammarly) and heavy generation, advanced detectors analyze the "cosine difference"—the distance in a multidimensional space between the original human text and the AI-edited version. This quantifies the degree of AI influence.

Pangram Labs uses an "active learning" loop to enhance its model. After an initial training, the model scans a massive corpus to identify its own errors (false positives/negatives). These hard-to-classify examples are then fed back into the training set, making the next version more robust.

Substack's CEO Chris Best is addressing low-quality AI content by focusing on transparency. The platform integrated Pangram's AI detector to combat "Clod Fishing"—the deceptive practice of passing off AI-generated text as authentic human work. The goal is to preserve the reader's expectation of a genuine human connection, not to ban AI tools altogether.

When a brand like Apple has a massive, stylistically consistent public corpus, LLMs become experts at mimicking it. This creates a paradox where new, human-written content is flagged as AI-generated because detectors recognize the perfectly emulated patterns they were trained on.

To stay ahead in the cat-and-mouse game of AI detection, Pangram internally develops its own "humanizer" tools—software designed to make AI text undetectable. They use these adversarial tools to train their detection model against the latest evasion techniques, without ever releasing the humanizers publicly.