Two photos, one person, a stated threshold.

The onboarding flow shipped with a face check. An applicant photographs a passport, takes a selfie, and the service answers. For the first month the answers looked clean. Then operations noticed that one applicant in every few dozen was being refused, and that the refusals clustered: older passports, one particular phone model, people photographed against a bright window. None of them were fraud. The comparison had not malfunctioned. It had been handed a question it could not answer, and the integration had no way to hear it say so.

That is the usual shape of a face match document verification problem. The model is rarely the weak part. The weak part is everything around the number it returns: the threshold nobody wrote down, the third outcome nobody allowed for, and the biometric nobody gave a deletion date.

Verification is one-to-one, and that decides the design

Face match answers a narrow question: is the person in this selfie the person pictured on this document? It compares one probe against one reference — a one-to-one verification. It is not a search. It does not ask who the person is, and it does not look through anyone else's face to find out.

The distinction is worth stating because three different face checks tend to get bundled into one "biometric step", and they fail in different ways.

• Presence — is a live person in front of the camera right now, rather than a print, a screen or a mask. That is liveness, and it can be established without keeping a face at all, which is the argument of the passive liveness piece.
• Identity — is that live person the one on the document submitted in this case. That is face match, and it is the subject here.
• History — has this face been seen before, under another name or on a refused application. That is a one-to-many search against a store you built, and it carries the obligations of a blocklist.

Liveness without face match lets a real person present someone else's document. Face match without liveness lets a photograph of the document holder pass as the holder. A history search is a separate product decision with a much larger privacy footprint, and it should never arrive as a side effect of the other two. Scoring the three separately, and recording them separately, is what lets a reviewer tell later which one actually failed.

The document portrait is the weaker image

The instinct is to worry about the selfie. It is taken on an unknown phone, in unknown light, by someone who may be in a hurry. But the harder image is usually the reference: the portrait printed on the identity document.

It is small — a few centimetres across, and reproduced at a fraction of the resolution the selfie has. It is printed, then laminated, then photographed again by the applicant, so it arrives as a copy of a copy. Security features sit on top of it by design: fine-line patterns, holographic overlays, laser engraving, a second ghost portrait. And it can be old. A passport issued nine years ago shows a face nine years younger than the one holding the phone.

Each of those degrades the comparison without making it wrong in any obvious way. The score simply drifts toward the middle, where the threshold has to decide something it has too little information to decide. That is why most of the useful engineering happens before the comparison runs.

• Exactly one face on each side — no face on the document, or two faces in the selfie, is not a low score. It is an input the comparison should never have received.
• A readable portrait region — glare across the face, a thumb over the corner or a crop that clips the chin are properties of this capture, and a retake fixes them.
• The document's age, where it is printed — the issue date is extracted anyway, and a large gap between issue and capture is context the reviewer should see before the score.

On the selfie side the list is shorter and more familiar: frontal pose, both eyes visible, even light, no heavy filter. Most of it can be checked on the device while the applicant is still holding it, which is the cheapest place to fix anything.

A score is not a decision

What a comparison produces is a similarity score. What the business needs is a decision. The step between them is a threshold, and the threshold is not a technical constant. It is a policy about which mistake you would rather make.

There are two errors, and the evaluation vocabulary for them is worth using precisely, because it is the vocabulary every benchmark report and every vendor datasheet is written in.

Raising the threshold lowers the false match rate and raises the false non-match rate. No setting minimises both, so the face match threshold is a statement of priorities and should be written down as one. "We use 0.8" is not a policy. A policy reads more like this: accept automatically when the score clears the threshold chosen for a false match rate of 1 in 10,000 on our own labelled pairs; this is threshold version 3, set on this date by this owner, and re-measured every quarter. The figures in that sentence are an example, not a recommendation. The structure is the recommendation.

Two phrases in it carry most of the weight. Our own labelled pairs, because a published error rate was measured on someone else's evaluation set, with its own capture conditions and its own population; the operating point that matters is the one on your applicants, your phones and your documents. And re-measured, because the population drifts, the document mix changes, and a threshold that was right at launch quietly stops being the one you chose. Public evaluations such as NIST's face recognition vendor tests have also documented that error rates can differ across demographic groups, which is one more reason to measure on the people you actually serve, broken down by group, rather than on an average.

Risk tiers can deserve different thresholds. A high-value account opening and a low-limit wallet top-up do not have to share an operating point, and forcing them to is how one of them ends up too loose or the other too strict.

Three outcomes, and the third needs a budget

A comparison that can only say match or no match will turn every input it cannot judge into one of the two — usually no match, because a degraded image scores low. That is what happened to the applicants at the top of this piece. The step had no word for "I could not compare these", so it said "these are different people".

The fix is a third outcome, and it has to be designed in rather than inferred from a middle band of scores. Cannot compare is a statement about the inputs — no usable face, a portrait under glare, a selfie too dark to read — and it goes somewhere different from a genuine low score. The inputs failed, so the right response is usually a retake, not a refusal and not a reviewer.

The liveness piece made the same case for presence: an inconclusive verdict is an honest one. The additional point for face match is that abstention has to carry a number. Declare an abstention ceiling — the share of attempts you expect to end in "cannot compare" — and watch it the way you watch an error rate. It tends to move for reasons that have nothing to do with the applicants.

• A capture change — a new camera screen, a new compression setting, a redesigned frame guide.
• A document mix change — a new market, a new issuer, a card format with a smaller or more heavily overlaid portrait.
• A device change — a popular phone model whose default processing smooths faces.

When the abstention rate rises, the capture flow broke. When the false non-match rate rises while abstention stays flat, the threshold or the population moved. Those are two different investigations, and you can only tell them apart if you counted them apart.

Review is a comparison, not a second opinion

Between the automatic accept and the clear refusal there is a band where a person looks. That band is where the threshold policy spends its human budget, and it is worth being honest about what the person can do there.

People are much worse at matching unfamiliar faces than they believe. Research on professional examiners — including a widely cited study of passport officers — found that trained staff make substantial numbers of errors when comparing a photograph with an unfamiliar person, and that years on the job did not reliably predict accuracy. A reviewer is not ground truth. A reviewer is a second comparator with different blind spots.

That changes how the review screen should be built.

• Both images at the same size — side by side, with the portrait enlarged from the source capture rather than shown as a thumbnail.
• Quality flags before the score — a reviewer who sees the number first is anchored to it; one who sees "glare on portrait, document issued 2017" is looking at the evidence.
• The same three answers — same person, different person, cannot tell, with the last one leading to a retake or a different document rather than a forced guess.
• Who decided, and why — the reviewer's answer belongs on the case record, next to the score it overrode.

The review band also produces the labelled pairs the threshold needs. Every decided case in it is one more measurement of where your operating point actually sits.

When the case carries more than one document

The Cogneris face match call takes one selfie and one or more identity documents in the same request — the documents field is repeatable, so a passport and a driving licence can be compared against the same capture at once. That is convenient, and it raises a question the integration has to answer before launch rather than during an incident: what does disagreement mean?

Three policies are defensible, and each has a cost.

• Every document must match — the strictest reading, and the one that turns a single poor portrait into a refusal of the whole case.
• The primary document must match — the others are supporting evidence, recorded but not decisive.
• Disagreement goes to review — the selfie matches one document and not the other, and that pattern is itself the signal.

The third deserves attention, because disagreement between two identity documents that claim the same person is rarely a quality problem. When the selfie matches the driving licence cleanly and the passport not at all, the likelier explanation is that one of the two documents is not what it says. That is a document-fraud question, and it belongs with the controls in the document fraud piece, not with the face threshold.

Where it sits in the flow

Face match is synchronous only. Zero-shot discovery, crop and split can each run as an asynchronous job; face match has no job operation, as the document tools reference states. The call is made while the applicant is present, and that is the right shape for it: the most useful thing a face check can do is ask for a retake while the person can still take one.

Three ordering choices follow.

• Check the inputs first — one face on each side and a readable portrait, before spending a comparison on images that were never going to be comparable.
• Retake on bad inputs, not on low scores — letting an applicant retry until the score clears the threshold turns the threshold into a suggestion.
• Cap the attempts and keep them all — the number of retakes and every score along the way are part of the case, because a sequence that climbs slowly toward the threshold looks different from one that clears it at once.

The same file limit as the rest of the document API applies to each image — 10 MB per file — and the same envelope comes back, so code that already checks hasErrors and reads meta.messages on an extraction reads a face match response the same way.

The biometric keeps the shortest clock

A face image used to verify identity is biometric data, and most privacy regimes put it in their most protected class. The GDPR treats biometric data processed to uniquely identify a person as a special category under Article 9. Brazil's LGPD lists biometric data among sensitive personal data. Illinois' BIPA requires a written retention schedule and destruction once the purpose is served. The details differ by jurisdiction; the design conclusion does not. Of everything in the case, the face should be the first thing to go.

The move is to separate what the case has to prove from what the comparison had to compute. The comparison needed two images. The record needs much less.

• The outcome — match, no match or cannot compare, and whether a person decided it.
• The score and the policy version — so the decision can be re-read against the threshold that made it.
• The quality flags — what the inputs looked like, in words rather than pixels.
• Hashes of the inputs — enough to prove later which images were compared, without keeping them.

The images themselves get the shortest retention class in the schedule, and a legal hold is what extends it for one specific case — not a default that extends it for all of them. The mechanics of per-class clocks and holds are in the retention piece; the point here is only that the face is the class to tune first.

The same question applies to whoever runs the comparison. Before a face check goes live, ask where the comparison is computed, which parties see the images on the way, and how long each of them keeps them — and get the answer in writing, because a retention promise that stops at your own storage bucket does not describe everywhere the face has been.

The six-line review

A face check that was designed rather than switched on can answer six questions without anyone opening the code.

• What is the threshold, and who owns it — a version, a date and a name, set against a stated false match rate on your own pairs.
• What is the abstention ceiling — the share of attempts expected to end in "cannot compare", and the alert when it is crossed.
• What does review see first — the images and the quality flags, not the score.
• What happens when two documents disagree — a written policy, and a route to fraud review rather than to the face threshold.
• How many retakes are allowed — and whether every attempt is kept on the case.
• When is the face deleted — a date per class, shorter than anything else in the case, with a hold as the exception.

The applicants at the top of this piece were never a fraud problem. They were a missing answer — cannot compare — and a threshold nobody had written down. Both were cheaper to add before launch than to explain afterwards.