Can AI invent fake references? Ours can't — by construction

A generative model asked for sources on a topic will produce a list. Some of the entries will be real, some will be real papers with the wrong year or journal, and some will be entirely fictional — correctly formatted, plausibly titled, and describing research that was never done. Anyone who has had this happen to them knows the entries look exactly like the real ones.

So the reasonable question about a tool that uses AI on reference lists is: what stops yours doing the same thing?

The answer is not a filter and not a confidence rule. It is that the operation which produces fabricated references — generate a plausible source — is not implemented. There is no code path that turns model output into a reference entry.

What the model is actually asked

Every reference the product shows you already exists in one of two places: your own document, or a bibliographic record retrieved by a lookup. The model's job is to judge and correct what a parser read, not to supply what is absent.

That constraint is written into the instruction the model receives, in those words:

"You are given entries already parsed from the reference
 list of a thesis. The PARSER split each entry into fields
 and it makes mistakes: judge how well each entry was read,
 and correct what you can.

 You are NOT checking whether the work exists, and you must
 NEVER invent bibliographic data. Every correction must be
 recoverable from the raw text of the entry itself. If the
 raw text has no title, the title is missing — do not
 supply one."

Read the last sentence again, because it is the load-bearing one. A missing title stays missing. The model is not permitted to fill a gap from its own knowledge even when it is confident, even when it would probably be right — because "probably right" is the exact mechanism that produces a fabricated reference in the first place.

Every correction must be recoverable from the raw text. A model that may only rearrange what is in front of it cannot conjure a source that is not.

So where does a reference come from, when one is missing?

From a lookup, against real bibliographic databases — the same records a librarian would consult. When your thesis cites something your bibliography does not list, finding a candidate entry is a search problem: query the identifying details against Crossref, OpenAlex, PubMed and the rest, and return what actually comes back.

A lookup has a failure mode a generated answer does not: it can come back empty, and that is the point. Finding nothing tells you something true, where a generated answer would have produced a confident, well-formatted entry rather than admit the gap.

What the AI is for, at each of the three phases

Finding a reference your bibliography does not contain runs in three phases, and it is worth being exact about the AI's job in each, because "AI-powered" usually conceals precisely this.

Phase I    guides the SEARCH
           what is this citation about · which keywords ·
           which engines are worth asking
           → produces a query. Not a reference.

Phase II   gathers from the engines
           every hit from every engine that answered,
           collated, nothing discarded, nothing judged
           → produces a pool of real records.

Phase III  screens, ranks, and gates
           deterministic screens bracket the model, which
           may only choose an INDEX from that pool and
           write a justification for the choice
           → produces one record from the pool, or none.

At no point does the model supply the answer. It shapes
the question, then chooses among what came back.

That middle column is the part that matters. A model asked "what was published on this in 2022?" can answer from nothing at all. A model asked "which of these six records that Crossref and PubMed just returned is the one this citation means?" cannot — the six records are the only things it can point at, and it has no mechanism for adding a seventh.

The rule the code states for this is blunter than anything we would write in marketing: the AI is overarching, but every proposal comes from the engines, not the AI. And in the ranking call itself, the candidates must already be link-validated before the model is allowed to see them at all.

Which is why the honest description of the pipeline is deterministic. Not because no model is involved, but because every artefact that reaches you came from a record an engine returned and a screen accepted. The model guides keywords, routes to engines, ranks a validated pool and explains its pick. There is no step at which it writes a source.

The one place the model does change your text

It would be dishonest to stop there, because the model does propose changes: corrected author splits, a corrected year, a corrected title where the parser ran two fields together. Those corrections are model output entering bibliographic data, and they deserve a straight account.

Three things constrain them. Each correction must be recoverable from the raw entry, as above. The original value is retained beside the proposal, so you see precisely what is being changed from and to. And nothing is applied until you click — the ledger tracks proposals, not edits.

The guard we removed on purpose

Here is the part that shows where the line really sits, because we got it wrong first and the fix is instructive.

There used to be a hard lock on years. If the model proposed a year different from the one hooked out of the text, and that year did not appear literally in the snippet, the whole row was discarded. It was built for a real case: a citation hooked as "Ministry of Economy (2018)" against which the model proposed a differently-named body and a different year — a different work entirely.

In the text     …as reported in the national survey (2620)…
Clearly         a typo for 2020
Model proposes  year 2020

Under the lock  2020 does not appear literally in the snippet
                → whole row discarded, silently
                → the person who could have fixed it in one
                  glance never saw it

A corrected year is, by definition, not the year written in the text. So the lock rejected every genuine correction — transposed digits, a year written one way in the citation and another in the reference, the ordinary human errors this product exists to catch — in order to prevent a case that a later stage already handled.

Because the protection was redundant. A screened row does not become a citation on the model's say-so; it becomes a suggestion, and a person has to accept or reject it. The lock was not defending the document. It was deciding, before any human saw the row, that a correction they would have recognised at a glance was not worth showing them.

So it was removed and replaced with disclosure: a row whose year the model changed now carries the original year beside it, and the person accepting it sees exactly what is being changed. Not silence — visibility.

Why not simply forbid the model from touching anything?

Because the errors people actually need caught are exactly the ones a strict lock refuses. "Nelson SD, W." is a real parser failure we have written about at length — an author name mangled into something that matches nothing. Fixing it requires proposing a different author split from the same raw string, which is precisely the operation a maximally cautious rule would forbid.

The distinction that matters is not whether a model may propose anything. It is whether it may propose something not present in the source material. Rearranging "Nelson SD, W." into a plausible author list uses only the characters already there. Supplying a title for an entry that has none would not, and is refused.

What this does not claim

It does not claim the model is always right. A proposed correction can be wrong, which is why the original is shown beside it and why acceptance is a human act rather than a default.

And it does not mean an unresolvable reference is evidence that anyone fabricated anything. This matters, because a post about fake references invites exactly that inference. In practice an entry that fails to resolve is nearly always a typo in an identifier, an unusual venue, a preprint that moved, or a parser that read the entry badly — not misconduct. The right response to "unresolved" is to look it up, not to draw a conclusion about the person who cited it.

It does not claim your reference list is verified as true. Confirming a reference resolves to a real record is not the same as confirming the source says what you cited it for — that judgement is yours, and no system we know of can make it.

And it does not claim AI is absent. It is used, on bounded extracts, for the decisions that genuinely need judgement. The claim is narrower and checkable: there is no operation in this product that asks a model to produce a source, so there is no mechanism by which one can appear.