What is an AI hallucination?
Short answer
An AI hallucination is output from a language model that is fluent and confident but factually false — an invented statistic, event, quote, case, or source, presented exactly as if it were real. The defining feature is that nothing about the text signals the error: hallucinated content reads precisely like accurate content.
The word is borrowed from psychology and it is slightly misleading. A person who hallucinates perceives something that isn’t there; a language model isn’t perceiving anything at all. It is producing the most plausible continuation of your prompt, and most of the time the most plausible continuation is also true. A hallucination is what you get when those two things come apart.
That is why hallucinations are dangerous in a way ordinary errors are not. A typo announces itself. A hallucinated reference arrives correctly formatted, with a plausible author list, a believable title, a real journal’s name, and a page range — every surface signal of legitimacy, attached to a paper that does not exist.
Hallucination is a property of every large language model in production today. Vendors reduce it with retrieval, citations, and training; none has eliminated it, and the honest ones say so.
Why do AI models hallucinate?
Short answer
Because a language model is trained to predict plausible text, not to know facts. When the model has no reliable knowledge for the answer, the training objective still rewards producing a confident, well-formed one — so it fills the gap with the most statistically likely invention.
A language model learns, from an enormous corpus, the patterns of how text continues. That objective produces astonishing fluency, and it has a blind spot: the model is never directly rewarded for saying “I don’t know”. Recent research from the model makers themselves frames it bluntly — standard training and evaluation reward guessing over acknowledging uncertainty.
This is why hallucinations cluster exactly where you would predict: specific numbers, niche topics, precise citations, names and dates — places where the training data is thin but the surrounding pattern (what a citation looks like, what a statistic sounds like) is strong. The model knows the shape of an answer perfectly and the substance of it not at all.
It is also why asking the model to double-check itself only partly helps. The same machinery that produced the invention evaluates it, with the same blind spot. Genuine verification has to come from outside the model — which is the theme of the rest of this guide.
What AI hallucinations look like in the wild
Short answer
The best-documented public example: in 2023, lawyers in the US case Mata v. Avianca filed a brief containing six court cases suggested by ChatGPT. None of the cases existed. The court sanctioned the lawyers — who said they had believed the citations were real.
The Avianca case is worth studying because everyone in it behaved normally. The tool produced authoritative-looking citations, complete with docket numbers and quotes. The lawyers read output that looked exactly like legal research. The failure wasn’t gullibility about weird text — it was that hallucinated text carries no visible marks.
The same pattern repeats across domains: research summaries citing papers whose authors are real but whose paper is not; biographies blending two people with similar names; confident statistics with no origin; “direct quotes” nobody ever said; product documentation for features that were never shipped.
Notice what these have in common: each one is checkable in principle — the case, the paper, the quote either exists or it doesn’t — but expensive to check by hand, one item at a time. That asymmetry, cheap to fabricate and tedious to verify, is the practical problem this guide ends on.
| Hallucination type | What it looks like | Checkable? |
|---|---|---|
| Fabricated citation | A perfectly formatted reference to a paper or case that does not exist | Yes — mechanically |
| Misattributed source | A real source cited for a claim it never makes | Yes — read the source |
| Invented statistic | A precise number with no traceable origin | Partly — provenance is checkable |
| Blended entities | Two similar people or companies merged into one biography | Yes — with effort |
| Confident non-fact | A plausible event or “fact” that simply never happened | Hardest — proving a negative |
How often does AI hallucinate?
Short answer
There is no single honest number. Measured rates vary enormously by model, task, and prompt — from a few percent on well-covered factual questions to a majority of citations in some early legal and academic benchmarks. Any page quoting one universal “hallucination rate” is overselling.
You will find pages claiming AI hallucinates “3% of the time” or “27% of the time”. Read the method behind any such number and it always describes one model, on one task, on one date — usually summarisation, where rates are lowest. Citation-generation tasks measure far worse, because references are precisely where plausibility and truth diverge hardest.
The rate also moves with every model release, in both directions: newer models hallucinate less on some benchmarks and more on others, and retrieval-augmented setups shift the failure from invention to misattribution — the source exists but doesn’t say what the model claims.
Our position, consistent with how we publish our own detection numbers: a rate without a method, a corpus, and a date is marketing, not measurement. The useful question is not “how often do models hallucinate in general” but “are the specific claims and citations in the document in front of me supported” — which, unlike the general question, you can actually answer.
The one hallucination you can actually check
Short answer
Fabricated and misattributed references. A citation either resolves to a real source or it doesn’t; the source either is what the document claims or it isn’t; it either supports the sentence it is attached to or it doesn’t. All three checks are mechanical — no judgement about “how AI-ish the prose sounds” required.
Most hallucinations are hard to catch because disproving a confident non-fact means proving a negative. Citations are the exception. They make a specific, binary, checkable claim: this source exists, is this thing, and says this. That is why fabricated references are both the most consequential hallucination in professional documents — they are the load-bearing evidence — and the most tractable one.
This is exactly what our reference checker does with a report: it follows every citation, confirms it resolves, confirms it is the source the document says it is, and checks whether it supports the specific claim it is attached to. In deep mode it goes further and tests whether each claim is entailed by the available evidence. Internal arithmetic is recomputed on the way through.
The result is coverage, not truth: a report whose citations all hold up can still be wrong, and the tool says so on every run. But a report whose citations don’t exist has told you something decisive — and unlike style-based AI detection, that finding is not a probability. The reference is fake whether a human or a model invented it.
How to protect your work from AI hallucinations
Short answer
Treat every specific, checkable claim in AI-assisted writing as unverified until a source outside the model confirms it — and check the citations first, because they are the cheapest thing to verify and the most damaging thing to get wrong.
Practical order of operations for anyone reviewing an AI-drafted report, memo, or submission:
- Check the references first. Every citation should resolve, be the claimed source, and support its sentence. This is mechanical — automate it.
- Chase the numbers. Any precise statistic needs an origin you can open. “Studies show” with no study is a finding, not a style problem.
- Watch the blends. Names, cases and companies that feel almost right are the signature of merged entities — verify identity, not just existence.
- Never use the model as its own fact-checker. Verification has to touch something outside the text: a resolvable source, a recomputed figure, a registry.
- Keep the human call. A clean citation check means the evidence holds up — it does not mean the argument is right. Coverage, not clearance.
Frequently asked questions
What is an AI hallucination in simple terms?
It’s when an AI states something false as if it were true — an invented fact, quote, statistic, or source — in completely fluent, confident language. Nothing about the text looks wrong; that is what makes it a hallucination rather than an obvious error.
Why does AI hallucinate?
Language models are trained to produce plausible text, not to know facts, and training rewards a confident answer over an admission of uncertainty. Where the model’s knowledge is thin but the pattern of an answer is strong — citations, statistics, names — it fills the gap with the most plausible invention.
Does ChatGPT hallucinate?
Yes — every large language model in production does, including the best ones. Vendors reduce hallucination with retrieval and training changes, and rates differ between models and versions, but none has eliminated it. The documented Mata v. Avianca case, where six ChatGPT-suggested court citations turned out not to exist, is the canonical example.
How often do AI models hallucinate?
There is no single honest rate. Measurements range from a few percent on well-covered factual tasks to far higher on citation generation, and every number is specific to one model, one task, and one date. Treat any universal percentage with suspicion, and treat the document in front of you as the thing to verify.
Can AI hallucinations be detected?
The checkable kind can. Fabricated and misattributed citations are mechanically verifiable — a reference resolves and supports its claim or it doesn’t — and that is what our reference checker automates. General fact-level hallucination is harder: in deep mode the checker tests whether claims are entailed by available evidence, but that is coverage of the evidence, not a truth verdict.
Is a hallucination the same as AI-written text?
No — they are different questions. AI-written text can be entirely accurate, and human-written text can be full of fabrications. Style detection asks who probably wrote it; hallucination checking asks whether what it says holds up. For documents that matter, the second question is usually the one with consequences.
How do I check if a citation is fake?
Try to resolve it: search the title, the DOI, the docket number. A fabricated reference typically fails at the first step — nothing with that exact identity exists — or resolves to a real source that never makes the cited claim. Doing this by hand works for one reference; for a whole report, automate it and read the exceptions.
Can I stop an AI from hallucinating?
You can reduce it — ask for sources, use retrieval-backed tools, keep questions inside well-documented territory — but you cannot switch it off, and asking the model to verify itself uses the same machinery that produced the error. Reliable verification is external: resolve the sources, recompute the numbers, and keep a person on the final call.
Sources and further reading
- 01OpenAI — Why language models hallucinate (the vendor’s own account of the mechanism)
- 02Wikipedia — Hallucination (artificial intelligence), incl. the Mata v. Avianca case
- 03Stipple reference checker — the citation checks described in this guide
Educational guidance, not a forensic certification. Detection technologies and standards change; review material decisions against current evidence.