What is an AI fact checker?
Short answer
An AI fact checker is software that verifies the checkable parts of a document mechanically: whether each citation resolves to a real source, whether that source supports the claim attached to it, whether the document’s own numbers reconcile, and whether claims contradict the evidence cited for them. It automates the legwork of fact-checking — not the judgement.
The phrase carries two meanings now, and both matter. The first is using AI to help check facts — software that follows references, reads sources, and recomputes figures far faster than a person. The second is fact-checking the AI itself: verifying reports, summaries and research that a language model produced, because fluent text and reliable text are not the same thing.
In practice the two collapse into one workflow, because the documents most in need of mechanical checking are the ones AI wrote. A model can produce a confident, well-structured report with fabricated references in minutes; a person armed only with a browser needs an hour to check ten of its citations. Tooling closes that gap.
What no tool does — ours included — is certify truth. Mechanical checking verifies the scaffolding: sources exist, sources match, numbers add up, claims are not contradicted by their own evidence. A report can pass every one of those checks and still be wrong. The honest job of an AI fact checker is to tell you where to look, and to make the scaffolding failures impossible to miss.
How often AI reports cite sources that do not exist
Short answer
In our published benchmark, 38.6% of citations in AI-generated research reports did not exist as cited (95% interval: 29.7% to 48.4%). On a heavily published topic the rate was 24.5%; on a thin topic it was 54.2% — the fabrication rate roughly doubled when real sources were scarce.
This is the reason the workflow below starts with citations. We followed 101 citations from AI-written reports to their sources and recorded what we found: 38.6% did not exist as cited. The number is not a curiosity about one bad model — the reports read fluently, the references were formatted correctly, and nothing on the surface separated the real citations from the invented ones.
The topic effect matters most for anyone using AI research tools on niche subjects: on a well-published topic, roughly one citation in four failed; on a thin topic, more than half did. The less real material exists, the more the model fills the gap with plausible fabrication — which means AI research is least reliable exactly where you can least verify it from memory.
The practical conclusion: citation checking is not a nice-to-have layer on AI-generated work. It is the difference between a document you can forward and a document you cannot.
What can be verified mechanically — and what cannot
Every fact-checking decision splits into a mechanical layer and a judgement layer. Knowing which is which tells you what to automate and what to read yourself.
| Layer | Checkable by software? | How |
|---|---|---|
| Citation exists | Yes | Follow the reference: does it resolve to a real page, paper, or record? |
| Citation supports the claim | Yes, substantially | Read the source against the claim it is attached to — a match check, not a truth check. |
| Internal arithmetic | Yes | Recompute totals, percentages, and carried-forward figures from the document’s own numbers. |
| Self-contradiction | Yes, substantially | Flag claims the document’s own cited evidence contradicts. |
| Claim is true of the world | No | Truth needs context, recency, and judgement. Software narrows where to look; a person decides. |
| Framing and omission | No | A report can be accurate line-by-line and still misleading. Only a reader catches this. |
How to fact-check an AI report, step by step
Short answer
Check the citations first, recompute the numbers second, then read the claims the checks flagged. If the references do not hold up, stop — the report has told you what you need to know about the care behind it.
This is the workflow we built our own tool around, and it works with or without it. The order is deliberate: the cheap, decisive checks come first.
| Step | What to do | Why this order |
|---|---|---|
| 1. Inventory the claims that matter | Mark the specific, consequential claims — figures, named findings, quoted sources. Ignore the connective prose. | Fact-checking everything is how nothing gets checked. |
| 2. Run the citations | Follow every reference behind a marked claim: does it exist, and does it say what the report says it says? | Fastest kill-shot. In our benchmark, over a third fail here. |
| 3. Recompute the arithmetic | Check totals, percentages, and any figure carried between sections. | Numbers that do not reconcile are objective, arguable-with-nobody errors. |
| 4. Read the flagged claims in context | For anything unsupported or contradicted, read the surrounding source material yourself. | This is where judgement enters — software got you here quickly. |
| 5. Decide with the stakes in mind | Forwarding to a colleague needs less certainty than publishing or acting. | Match the depth of checking to the cost of being wrong. |
Fact-checker websites vs citation checkers: which to use when
Short answer
Human fact-checking sites like Snopes, FactCheck.org and PolitiFact verify public claims with editors and evidence. Mechanical checkers verify the references and arithmetic inside your document. For an AI-written report you usually need the mechanical kind first — its problems are in the scaffolding, not the headlines.
Searches for “fact checker websites” usually want one of two different things, and using the wrong kind wastes the afternoon.
| Resource | What it checks | Best for |
|---|---|---|
| Snopes / FactCheck.org / PolitiFact | Public claims, rumours, political statements — researched by human editors. | Checking a specific public claim someone repeated. |
| Google Fact Check Explorer | A searchable index of published fact-checks from many organisations. | Finding whether a claim has already been checked, free. |
| Stipple fact-check (ours) | The citations, internal arithmetic, and self-consistency of a document you give it. | AI-written reports, whitepapers, and research before you act on them. |
| Originality.ai fact checker | Automated claim assessment aimed at web content teams. | Editorial workflows already on that platform. |
| University library guides | Teaching materials on evaluating AI output (many are excellent). | Learning the craft; not a tool, a curriculum. |
Is there a free AI fact checker?
Short answer
Yes. Google Fact Check Explorer is free for published claims. Stipple’s fact-check tool runs on free weekly credits with no card — paste a report or a public link and every citation is checked that it resolves and supports its claim. Most commercial tools offer at least a limited free path.
Free is genuinely workable here, because the mechanical layer of fact-checking does not need an enterprise contract. Our own tool’s free weekly credits cover the workflow above for a handful of reports; Google’s Fact Check Explorer costs nothing and answers “has someone already checked this claim?” instantly.
What to watch in any free tool, ours included: whether it shows its work. A fact-check result you cannot inspect — which source failed, which number did not reconcile — is a verdict you cannot defend to anyone else. The output you want is per-citation status and named findings, not a single trust score.
The limits — what “supported” does and does not mean
One definition carries this entire field, and most marketing blurs it: “supported” means a source backs the claim. It does not mean the claim is true. A report can cite a real study accurately and the study itself can be wrong, retracted, or outdated. Mechanical checking verifies the chain of custody between claim and source — the substance is still yours to judge.
The same honesty applies in reverse. “Unsupported” means no backing evidence was found — not that the claim is false. Thin topics produce unsupported-but-true claims constantly. Treat flags as a map of where to spend your attention, not as verdicts.
And the field’s hardest limit: no tool catches what a report leaves out. Software verifies what is on the page. Omission, framing, and emphasis — the ways an accurate document misleads — remain a reader’s job. Any AI fact checker that claims otherwise has left honesty behind.
Frequently asked questions
What is the best AI fact checker?
For AI-written reports and research, the best tool is one that checks the document mechanically — every citation followed and matched to its claim, arithmetic recomputed, contradictions flagged — and shows its work per finding. That is what Stipple’s fact-check tool does, disclosed as our own. For public claims and rumours, the established fact-checking sites with human editors (Snopes, FactCheck.org, PolitiFact) remain the standard.
Is there a free AI fact checker?
Yes. Stipple’s fact-check tool runs on free weekly credits with no card required, and Google Fact Check Explorer is free for searching published fact-checks. Prefer free tools that show per-citation results over ones that return a single unexplained score.
Can AI fact-check itself?
Partially, and with an important caveat. Software can mechanically verify an AI report’s citations, arithmetic and self-consistency — checks that do not depend on trusting the model that wrote the text. What AI cannot reliably do is verify the truth of its own claims; the same tendency that fabricates a citation can fabricate a confirmation. Mechanical checks plus human judgement is the honest architecture.
How often does AI make up citations?
In our published benchmark, 38.6% of citations in AI-generated research reports did not exist as cited (95% interval 29.7–48.4%). The rate was 24.5% on a heavily published topic and 54.2% on a thin one — fabrication roughly doubles when real sources are scarce.
What are the best fact checker websites?
It depends on the job. For public claims: Snopes, FactCheck.org, PolitiFact, and the international fact-checkers indexed by Google Fact Check Explorer. For the inside of a document — do its citations and numbers hold up — you want a mechanical checker like Stipple’s fact-check tool rather than a claims site.
Does a passing fact-check mean the report is true?
No. Passing means the scaffolding held: citations exist and support their claims, the arithmetic reconciles, nothing contradicts its own sources. The substance can still be wrong, outdated, or misleading by omission. Mechanical checking tells you where to look; it does not replace reading.
Can it check a report from just a link?
Yes — Stipple’s tool accepts a public http(s) link to a page or PDF and fetches it directly, as well as pasted text and uploaded files.
Sources and further reading
- 01Stipple — AI citation hallucination benchmark (101 citations followed to source)
- 02Google Fact Check Explorer
- 03IFCN — International Fact-Checking Network code of principles
- 04Duke Reporters’ Lab — global fact-checking census
Educational guidance, not a forensic certification. Detection technologies and standards change; review material decisions against current evidence.