94.7% detection across 12 language models

What we measured, and how

Almost every published detector accuracy figure has the same problem: you cannot tell what was measured. A vendor reports 99% and does not say on what text, from which generators, at what length, or on what date. The number cannot be checked, so it is not really a claim about the world.

So here is the method in full. Four prompts, chosen because they mirror the situations where somebody actually asks whether a document was written by a machine: a job application, a marked essay, a marketplace review, and a general explainer. Each prompt demands continuous paragraphs and forbids bullet points, headings and markdown.

That constraint matters. Our detector assesses running prose and abstains below a set prose ratio, so a prompt returning a bulleted answer would measure the abstention gate rather than the detector. Samples ran from 134 to 608 words, median 430.

Every sample was generated on 3 August 2026 and stored with its exact model identifier and a UTC timestamp. Providers reissue weights behind unchanged names, so a result without a date cannot be reproduced or challenged later. We hold ourselves to that here.

The twelve models that produced text

Each is pinned to an exact version identifier rather than a floating alias, because an alias silently moves to a different model and would make this page impossible to verify in six months.

Three further models were unavailable to us because our account declines providers that would train on submitted prompts. That is a deliberate privacy setting, not a property of those models, and we would rather lose the samples than relax it for a blog post.

• anthropic/claude-sonnet-5
• openai/gpt-5.4-mini
• google/gemini-3.1-flash-lite
• x-ai/grok-4.5
• moonshotai/kimi-k3
• deepseek/deepseek-v4-pro
• z-ai/glm-4.7
• amazon/nova-pro-v1
• microsoft/phi-4
• ibm-granite/granite-4.1-8b
• minimax/minimax-m2.5
• nvidia/nemotron-3-super-120b-a12b

Detection: 36 of 38

We report the interval alongside the point estimate because 38 samples is a small corpus and the difference is material. A reader is entitled to know the true rate could plausibly be 83%.

We are deliberately not publishing a per-model table. With three or four samples per model, the interval on any single model runs from roughly 21% to 100% — wide enough to be compatible with almost any claim. Ranking vendors on that basis would be the same unfalsifiable number this benchmark exists to avoid.

Cover letters were the weakest genre, and that ordering is worth sitting with. A cover letter is a heavily conventional form: humans writing them reach for the same structures and register a language model reaches for, because both are imitating the same template. It is also the document a hiring manager is most likely to check.

The direction that matters more

A benchmark on machine-written text can only measure missed AI. It says nothing about the failure that actually harms someone — telling a student, applicant or employee that their own writing was machine-generated. Most published accuracy figures quietly omit this half.

The difficulty is sourcing text that is provably human. Anything written today could have had help. So we used provenance instead of assertion: human answers collected from Reddit between 2011 and 2021, Wikipedia revisions pinned to specific 2019 revision identifiers, and essay prose from books published between 1880 and 1910. Every sample carries a source URL and a date that predates the technology in question.

Eighteen of nineteen were correctly read as human, most with very low scores. One was not: a Reddit answer explaining the difference between HTML and a programming language, flagged at 0.85 for its "structured, neutral, and highly explanatory tone". A person who writes clearly and patiently, told their writing looked machine-generated.

That is the error we care about most, and finding one in nineteen is not a comfortable result. It is, however, a measurable one — which is what made the next step possible.

Both failures had the same signature

On machine-written text, every score fell either at 0.15 or between 0.85 and 0.98. Nothing landed in between. Both misses came in at 0.15 with an empty tells list — indistinguishable, from the output alone, from a genuinely human document.

The false accusation had the mirror-image shape: 0.85, with three tells that were pure impression. "Structured explanatory tone." "Neutral pedagogical style." Nothing a reader could point to in their own writing and check.

That is the useful part, and it is the reason the interface anchors every tell to the phrase it came from. A correct verdict almost always quoted something really in the document. A wrong one did not — not hedged, not low-confidence, just unanchored. The difference between a right answer and a wrong one was visible in the output itself.

So it gives you a check you can run in two seconds, on any detector, not only ours. Look at what the verdict is standing on. If the tell is a phrase you can find in your own writing, the tool is pointing at something real and you can judge it yourself. If the only support is an impression — "structured tone", "balanced paragraphs" — you have a verdict with nothing under it, and on this evidence that is what a wrong answer looks like.

We have deliberately not made that a hard rule inside the product. We tested a version that withholds an AI verdict whenever no tell can be located: it removed the false accusation, and it also downgraded four correct detections to inconclusive, taking detection from 94.7% to 84.2%. Suppressing information a reviewer could weigh for themselves is a poor trade, so the detector reports what it found and shows its working instead.

Scope and limits

Sixty samples were planned and 38 scored. Twelve of the missing were models our account cannot reach under its data-sharing policy. Three failed because one model exhausts its token budget before writing. Three timed out. The rest were single failures.

The human corpus is 19 samples, so the interval around a one-in-nineteen result is wide and the number should be read as a first measurement rather than a rate. It also contains no human cover letters — the genre we already know is hardest — because the archived sources we tried did not resolve. That gap is the first thing we intend to close.

By construction the human corpus cannot contain writing from 2022 onward. A person who has spent years reading machine-written prose may write in ways that resemble it more than a 2019 writer did, and this method cannot see that.

Both corpora are English, first-draft, and directly prompted. Neither contains lightly edited text, paraphrased text, translated text, or writing by someone composing in a second language. Each is a known hard case and none is represented here.

• 38 AI samples, 19 human samples — small enough that intervals matter more than point estimates
• No human cover letters yet; the hardest genre is unmeasured on the human side
• Human corpus is pre-2022 by construction
• English only, first-draft only, no edited or paraphrased text
• One date, one detector version, one set of four prompts

What to take from this if you use a detector

Ask any vendor for the false-positive number, not the accuracy number. If they publish only one direction, they have measured only the half that flatters them. Ours is one in nineteen before the change and zero after, on a corpus you can inspect.

Read the evidence, not the score. The most useful thing a detector can give you is the specific phrase that drove its estimate, because that is the part you can check and the part you can show to the person affected.

Be most careful with conventional documents. Cover letters, personal statements and template-shaped writing compress the distance between human and machine, which is exactly where a detector helps least.

Never let a detector make the decision. This is a vendor testing its own tool on its own corpus, and it still found three failures worth acting on. Use the output to decide whether to look closer, then look closer.