How adverse-media and sanctions screening data is collected — the sources, the gaps, and the judgement

The sanctions lists, by name

The index is built from four primary sources. The US Treasury’s OFAC Specially Designated Nationals list, ingested twice: the primary names file and the alternate-names file, because OFAC’s own guidance warns that screening on primary names alone misses a large share of the list — an alias is exactly what a subject uses. In the current build 38,981 indexed names carry an OFAC source.

The EU consolidated financial sanctions list, published under an open licence and reformatted — 30,231 indexed names. The UK sanctions list — 13,603. And Australia’s consolidated list published by the Department of Foreign Affairs and Trade, the DFAT list — 10,778. The same person is often designated on several lists, so those memberships overlap: after normalisation and de-duplication the index holds 78,293 unique entries.

Taking the lists from the issuers rather than through an aggregator is not a purity preference. The best-known aggregator’s data is licensed for non-commercial use, and that restriction follows the data whether it is served from an API or downloaded in bulk. The issuer lists carry their own terms — public domain in the US, open licences elsewhere — and consuming them at the source is what makes the result usable inside a commercial product without a licence question hanging over every record. This is the configuration deployed on stipple.sh; the code also carries an optional path to a licensed aggregator, which is not enabled.

What is deliberately not in the index

The UN Security Council consolidated list is not ingested. Its general terms of use permit personal, non-commercial copying and restrict redistribution of derived compilations, which does not sit comfortably with shipping a compiled index inside a commercial product. The coverage cost was measured before deciding: of the names indexed, 52 — one in a thousand — came from the UN list alone, because UN measures are implemented into EU, UK and Australian law and the same designations arrive through those lists. The parser is kept so the decision is one line to reverse if permission is obtained.

There is no PEP data, and this is the absence that matters more. Politically-exposed-person screening is a regulatory obligation in many regimes, and no government publishes a consolidated PEP list; assembling one is precisely the value the commercial aggregators add, and their data carries the licence terms above. So the reply’s sanctions note reports that PEP screening was not performed, and its PEP flag stays false. It never reports “no PEP hits”, because it did not look, and a tool that let “no sanctions hit” read as “no PEP exposure” would be lying by omission.

Two other boundaries are worth naming. The index is a sanctions index — designations — not a law-enforcement wanted list, a court register or a disqualified-directors register. And the lists are the issuers’ lists as published; a designation the issuer has not yet published is, by definition, not in anyone’s index.

Dates: what fresh means per list

A screening record has to say how fresh its data was, and a single “last updated” date would be a lie in both directions. So each list carries its own date, and the reply states which kind of date it is.

For OFAC and the Australian list the date is the fetch date — the day the build pulled the file. For the EU and UK lists the issuer stamps a publication date on the file, and that is the date carried. At the build of 27 August 2026 the EU date read 5 August and the UK date read 3 June: not stale downloads, but lists that had not changed since those days.

That distinction is easy to get wrong in the alarming direction — a reviewer sees “June” in August and assumes a broken feed — and getting it wrong in the alarming direction is how a compliance tool loses trust. The dates travel with the result so the reviewer can judge for themselves.

How a name is matched

Every name in the index is normalised the same way as the query — case, accents, punctuation, transliteration noise — and the subject is scored against it. Below 0.70 a candidate is not surfaced at all, because near-misses on common names generate noise that buries the real hits. At most eight candidates come back, best first.

At 0.90 or above a candidate is a strong candidate — but only if a date of birth or a country corroborates it. If the only thing that agrees is the name, or the date of birth or country you gave does not agree, it comes back `possible` with the corroboration recorded, whatever the score. A perfect match on a string is still a match on a string.

When several uncorroborated candidates score above the strong threshold at once, the result raises a common-name flag rather than presenting the top one. The matching is deterministic — the same query against the same index returns the same candidates with the same scores — which is what makes a result reproducible six months later.

The news layer, step by step

The query is the subject’s name together with a fixed set of adverse terms — fraud, convicted, charged, arrested, indicted, bribery, corruption, money laundering, sanctions, embezzlement, terrorism, trafficking, investigation. It runs against a commercial web-search index; if that provider is unavailable, it fails over to the GDELT global news index, which is keyless, news-focused and roughly fifteen minutes fresh. The reply names which provider answered and lists the attempts when a failover happened, because “three articles read” means nothing to a reviewer who cannot tell where they came from.

Up to three results are read — fewer when a page is dead or blocked, and the reply’s `articles_read` says how many. Where the provider supplies an excerpt of at least 200 characters, that excerpt is used; otherwise the page is fetched, in parallel, with dead and blocked pages dropped. That excerpt path exists for a reason: a paywalled or bot-walled article used to vanish from the screen entirely — a silent recall loss on exactly the outlets that break financial-crime stories.

Then one batched judgement runs over all the articles at once: for each, is it about this subject or a different person who shares the name — using role, employer, country and age to tell them apart — and if it is the subject, which category of adverse content, at what severity (convicted, charged, alleged, mention), on what date, with a quoted evidence line of at most 160 characters. Articles judged to be about someone else are returned under `excluded` with the reason. Articles judged not adverse are dropped.

The caps are deliberate. At most three usable articles per subject, and on the news-index path English-language and recency-weighted. A wider net would find more and cost more and, above all, would be no less a statement about coverage. The reply says what it searched and how much; that is the honest unit of screening.

What a weekly rebuild promises

A scheduled weekly job downloads each list from its issuer, normalises and de-duplicates it, stamps each source with its date, and publishes the result. OFAC in particular changes most days, and a designation invisible for a fortnight is the failure that matters, which is why the cadence is weekly rather than monthly. If a refreshed index cannot be fetched, the copy shipped with the service keeps serving — and the dates in the reply are what show it.

Weekly is also a stated compromise rather than a boast. A designation published on a Tuesday can be absent until the following build, and the reply’s dates let a reviewer see exactly that window. Where the obligation demands same-day coverage, the honest answer is that this index is one input and the issuer’s own search is another.

Each build records its own timestamp in the index status, and the reply carries each list’s date and a one-line coverage statement — primary sanctions designations only, no PEP data — so the boundary is in the record, not only on this page.

What to record on your side

A stored “clear” ages into a claim nobody can support. A stored coverage record stays true: on this date, these four lists at these dates, this query against this provider, these candidates at these strengths, these articles read and these excluded. All of it is in the reply, and none of it needs to be reconstructed later.

Two habits follow. Keep the exclusions — they are the evidence that the question was actually considered. And relay the `limitations` paragraph as written: it says that this is coverage and not a determination, that a name-only hit is possible and never confirmed, and that nothing found is not a clean record. Those are the three sentences a compliance file most needs and most often lacks.