Deterministic first, AI second — and why the order is the whole design

Ask a language model to match your citations to your bibliography and it will do it, confidently, including the ones that do not match. Run the rules first and the model never gets the chance.

Every citation checker faces the same problem in the same order. You have a few hundred in-text citations and a few hundred reference entries, and something has to decide which goes with which. The interesting question is not whether AI is involved. It is where in the sequence it is allowed to speak.

Our answer: as late as possible, on the smallest remaining pile, and never over the top of an answer the rules already found.

What the rules can settle on their own

A surprising share of the work needs no judgement at all. Three passes, in this order, and each only sees what the one before it could not resolve.

  312 citations   340 reference entries

  PASS 1   a shared identifier
           a DOI in the text and the same DOI
           in the entry — the same object, named
           the same way, by both
                                    settled: many

  PASS 2   exact author and year
           surname as written, year as written,
           no interpretation applied
                                    settled: most of the rest

  PASS 3   the reference line as written
           the entry compared as a whole string,
           not as parsed fields
                                    settled: a few more

  --------------------------------------------------
  what is left is what the rules could not answer

Nothing in those three passes is clever, and that is the point. Each one either matches or does not, gives the same answer every time it is run, and can be explained in a sentence to somebody who does not care how software works. When a pass settles a pair, the pairing is final — there is no later stage that revisits it.

A rule that always gives the same answer is not a lesser tool than a model. It is a different kind of claim: one you can check.

Where it stops being arithmetic

What survives the cascade is the genuinely ambiguous residue, and it is small. Two entries by the same author in the same year. A surname transliterated one way in the text and another in the list. An organisation cited by acronym. A reference whose year is missing entirely.

These are not failures of the rules. They are cases where the document itself is ambiguous, and no amount of stricter matching resolves them — a stricter rule would simply produce a confident wrong answer instead of an honest unresolved one.

That residue is what the AI review is for. It sees the pairs the rules could not settle, in context, and offers a reading. Not a decision — a reading, which you accept or reject. We wrote separately about why a mangled author name is so often the thing that leaves a pair unresolved.

Why not simply ask the model first?

Because a model asked first answers everything, and once it has answered you have lost the ability to tell its guesses from its certainties.

Put three hundred citations and three hundred references in front of a language model and it will return three hundred pairings. Most will be right. Some will be right for the wrong reason. A few will be pairings it constructed because the shape of the task implied one existed. Nothing in the output distinguishes those cases, because the model produced them all by the same process and in the same tone.

Run the rules first and the shape of the problem inverts. The pairs the rules settled are settled by something you can inspect. What reaches the model is explicitly the hard residue — and its suggestions arrive labelled as suggestions, on a pile small enough that a person can actually read them.

  MODEL FIRST                    RULES FIRST

  300 pairings returned          most pairs settled by rule
  all in the same voice          each one inspectable
  right, right, right, wrong,    a small residue left over
  right, wrong, right...         and it is MARKED as residue

  you cannot tell which is       the uncertain pairs are the
  which without checking         only ones you have to read
  all 300 yourself

  confidence is uniform          confidence matches reality

There is a second, quieter reason. A deterministic pass costs nothing to run and gives the same answer in a year's time. A model changes — providers update them, and an answer that came from a model in January is not guaranteed from the same model in June. Anything load-bearing that can be settled by a rule should be settled by a rule, so that the parts of your report that must be stable are not resting on something that moves.

The hook that is not identity

One rule inside the cascade is worth stating plainly, because it looks like it should be stronger than it is: a surname and a year together are not an identity.

They are a hook — a cheap way to find candidates worth comparing. A thesis can easily cite two different papers by the same author in the same year, and a prolific author in a narrow field makes that ordinary rather than unusual. Treating the hook as proof would pair the wrong two things and never say so.

So the hook narrows the search and something else confirms it. When nothing confirms it, the pair stays unresolved and goes to review, which is a worse-looking result and a more honest one. The alternative — a report that pairs everything because pairing everything looks complete — is the outcome we spend the most effort avoiding. A related case is one author, two spellings, where the hook is right and the surname is not.

The review cannot overturn a settled pair

This is the constraint that makes the ordering more than a performance trick. The AI review is additive only. It reads the unresolved residue and may propose pairings within it. It is never shown the pairs the rules already settled, and it has no mechanism for reopening one.

The temptation to allow it is real. A model reading the whole document would sometimes spot a rule-matched pair that is wrong — a DOI that appears twice because an entry was duplicated, say. Letting it flag those would catch a genuine class of error.

It would also destroy the property the whole design exists for. The moment a model can revisit settled pairs, every pair becomes a model output again: to trust any of them you must now trust the review that declined to reopen them. The uniform confidence we eliminated by running rules first comes straight back, one layer up.

A guarantee you can only verify by trusting the thing it guards is not a guarantee.

So the wrong duplicate DOI is caught elsewhere — by the de-duplication pass, which is itself deterministic and runs before any of this. Each problem gets a mechanism suited to it, and none of them is "let the model look at everything and tell us".

What the split means for what you read

The practical effect shows up in the report. A pair matched by a shared identifier and a pair proposed by the AI review are both matched pairs, but they are not the same kind of claim, and the report does not flatten them into one. Each pair records how it was matched.

That means you can read the document at two speeds. The rule-matched pairs you can scan; they are as reliable as the identifiers they rest on. The AI-reviewed pairs deserve your attention, and there are few enough of them that giving it is realistic.

A checker that reported one undifferentiated list of matches would be easier to build and would tell you less. The split is the information.

What this does not claim

None of this makes the deterministic passes infallible. A DOI typed wrongly in the text matches nothing, and no cascade recovers a character that was never there. What the order buys is not perfection — it is that the failures are visible as failures rather than disguised as confident matches.

Nor does running rules first make the AI review optional. Without it the residue would simply stay unresolved, and a report that ends with a hundred shrugs is not much use to anyone. The two halves are not rivals; one settles what can be settled and hands over a short, honest list of what cannot.

And, as with everything else in the product, the output reports referencing quality and nothing more. An unresolved pair is not a finding of misconduct; it is most often a transliteration, a repeated author-year, or a reference list assembled across four years by a person who was also writing a thesis. The order of operations exists to make that report worth reading — not to make it look severe.