Not every page deserves a frontier model.

One model for every page is a pricing decision

Take a mixed intake — supplier invoices, delivery notes, bank statements, insurance certificates, the occasional forty-page contract — and look at what the pipeline actually does with it. A large share of those pages are born-digital PDFs with an intact text layer, a stable layout and six fields worth reading. A smaller share are photographs taken at an angle in a warehouse. A smaller share still are dense tables that span pages and only make sense read together. If every one of them goes to the same model, the cheap pages subsidise nothing and the expensive pages are underserved: you are paying frontier prices to read a clean header, and you are giving the genuinely hard document exactly the same single pass you gave the easy one.

That is the argument for routing, and it is a good one. But notice what it is not. It is not "use a smaller model to save money", which is a procurement move that trades accuracy for unit price and shows up two quarters later as rework. Routing is the claim that the correct amount of compute varies per document, and the variation is predictable. Everything difficult about it follows from the second half of that sentence, because a router is a prediction, and a prediction that is wrong on the expensive cases costs more than never having routed at all.

The reason this matters more in 2026 than it did in 2024 is that the spread between tiers got wide. Reading a page with a text layer and a deterministic parser and reading it with a reasoning model that thinks before answering are not two points on a smooth curve; they differ by orders of magnitude in cost, by a second or more in latency, and by where the work is allowed to run. When the spread is that wide, the routing policy is not an optimisation detail. It is the largest single lever on what a finished result costs, and it is also, unhelpfully, a lever that moves accuracy.

The four decisions people call "routing"

"Inference routing" gets used for four different decisions that have different inputs, different failure modes and, in most teams, different owners. Conflating them is why routing discussions go in circles.

Only the first two are about model selection. The third is mostly a deployment and data-protection question, and the fourth is the one that decides whether the other three are safe to make automatically. Teams that build a router usually build the first, occasionally the second, and discover the third and fourth in production.

Route on evidence, not on document type

The intuitive design is to classify the document — invoice, statement, certificate — and look up a model per class. It is intuitive and it is a weak predictor, because the thing that makes a page expensive to read is rarely what the document is. It is how it arrived. Two invoices from the same supplier, one exported from an accounting system and one photographed off a printer, are the same document type and completely different inference problems.

The signals that actually predict required effort are properties of the artifact and of the job, and most of them cost almost nothing to compute before any model runs.

The constraint that disciplines this list: the router must be cheaper than the decision it makes. A classifier that needs a vision model to decide whether to use a vision model has spent the saving before it exists. In practice the useful routers are boring — file inspection, a handful of image statistics, a lookup against sender history, and a policy table. Boring is also what makes them explainable, which matters later when someone asks why two identical-looking documents were processed differently.

Field risk deserves its own note, because it is the signal most routers ignore and the one buyers care about most. Effort should be allocated per field, not per document. A page can be trivially readable in nine fields and consequential in one — the payable amount, the bank details, the expiry date on a certificate. Routing the whole page to the cheapest tier that handles the easy nine is how the tenth becomes an incident. The workable pattern is to read the page at the tier its evidence justifies, then re-read the high-risk fields at a higher tier when the cheap read cannot be corroborated.

What each tier is honestly for

Vendors describe their tiers by capability. It is more useful to describe them by how they fail, because that is what determines where they belong in a ladder.

Two rules fall out of that table. The first is that escalation must be monotone in cost — each step up should strictly add capability, so a failure at one tier is informative about the next. A ladder where two tiers are merely different rather than ordered gives you no principled way to decide which to try, and you end up running both.

The second is stricter: a tier that cannot abstain cannot be at the bottom of a ladder. The whole economic case for routing rests on catching the cheap tier's mistakes. If the cheap tier always returns something and never says it could not read the field, you do not learn that it failed — you learn it three weeks later from a counterparty. This is the same discipline as letting the extractor decline to answer, and routing raises the stakes on it, because abstention is not just an honest output any more. It is the control signal the whole architecture runs on.

Escalate on a failed check, not on a confidence score

The obvious escalation trigger is confidence: if the cheap model is unsure, try the expensive one. It is obvious and it does not work, for a reason worth being precise about. Confidence scores are not comparable across models. A classical OCR engine's per-character score, a vision model's token probability and a reasoning model's stated certainty measure different quantities on different scales, and none of them is calibrated against your acceptance criteria. A threshold tuned on one tier does not transfer to another, and it does not survive the model behind the tier being updated.

What does transfer is a deterministic check, because it is defined in your terms rather than the model's. Line items that do not sum to the stated total. A required field that is absent. A tax identifier that fails its checksum. A date outside the contract period. A value that contradicts the purchase order or the goods receipt. A currency the agreement does not cover. These are facts about the output, they mean the same thing regardless of which tier produced it, and they are the correct escalation trigger: this result did not survive validation, so spend more.

Abstention is the second trigger, and it is the cheaper one. A tier that returns "the amount on page 2 is not legible" has told you exactly which higher-tier read to run and on which region of which page, which is far more efficient than re-processing the whole document. Confidence still has a use, but a narrow one: as a tie-breaker inside a tier, on a threshold calibrated per field against observed error rates, not as the thing that decides how much money to spend next.

Three properties keep the escalation ladder from becoming its own problem. It needs a depth ceiling, because unbounded retry is how a per-page cost becomes a per-page range nobody budgeted. It needs the escalation to carry its reason forward, so the higher tier is told which check failed and on which field rather than being handed the document again from scratch. And it needs a terminal state that is not success — when the top tier also fails validation, the correct output is an exception with the failed check attached, not the best guess available. A ladder whose last rung returns a guess has converted a detectable failure into an undetectable one, which is worse than the cheap tier alone.

Placement is constrained before it is optimised

Choosing where inference runs looks like a cost and latency question, and for a single-tenant workload in one jurisdiction it is. For most enterprise document workloads it is a constraint problem first, and the constraints do not negotiate.

The document is the payload, and documents carry personal and commercial data. Routing a page to a model in another region, or to a provider not named in the contract, is not a cost decision the router gets to make on price — it is a change to who processes the data, with a transfer question attached. A router that treats providers as interchangeable capacity will, on the day a preferred endpoint is saturated, cheerfully commit a compliance breach to preserve a latency target. The correct design makes residency and permitted-provider constraints a hard filter applied before any cost comparison, and makes the empty result set a queue rather than a fallback.

Within what is permitted, the real trade is between utilisation and latency. Accelerator economics reward batching — larger batches amortise loading and keep the device busy — but batching means waiting, and waiting is not available when a person is on a screen expecting an answer. This is why latency class belongs in the routing signal set. The same document from an overnight reconciliation run and from an interactive onboarding flow should not take the same path: one can wait for a full batch on a well-utilised device, and the other cannot. Teams that route only on content end up sizing their entire fleet for the interactive worst case and paying for the idle time.

Two smaller things that dominate real bills more than model choice does. Cold starts: a rarely-used specialised model that must be loaded before it runs can cost more in wall-clock than the general model it was meant to be cheaper than, which is an argument for fewer tiers rather than more. And the fixed floor of the surrounding work — file handling, rendering, storage, egress, the checks themselves — which does not shrink when the model does, and which sets a hard bound on how much routing can save. Below a certain volume that floor is the whole bill, and the compute line is not where the money is.

Cost per accepted field, not cost per page

Routing decisions are usually justified with a per-page price comparison, which is the wrong unit because it prices the attempt rather than the outcome. The number that makes routing legible is the expected cost of a field that survives validation, and it has four terms: the cost of the tier you tried, the probability it produces an accepted result, the cost of the escalation when it does not, and the cost of the failures that are not caught — rework, an exception that leaves the building, or a wrong value entering a system of record.

Write it out for a single field and the structure of the decision becomes obvious. Routing down is worth it when the cheap tier resolves most of the volume and its failures are reliably detected by a check. If detection is perfect, routing down is close to free: the worst case is that you paid the cheap tier's price twice over before landing on the answer you would have got anyway. If detection is partial, the undetected share is multiplied by a cost that is not a compute cost at all, and that term dominates everything else in the expression. This is the whole reason the failure path is a first-class routing decision rather than an implementation detail.

Two consequences worth stating plainly. A cheap tier with weak detection is more expensive than the frontier tier, and no pricing page will show you that. And the value of routing is bounded above by the share of your volume that is genuinely easy — if 20% of your pages are clean born-digital files, the ceiling on the saving is what those 20% would have cost at the top tier, minus the cost of the router and the checks. Measure that share before building anything, because for some intakes the honest answer is that there is not enough easy volume to pay for the machinery.

The routing policy is a control-plane object

Here is the property that makes routing different from ordinary performance tuning: changing the routing policy changes the output for the same input, with no code deployed. Move a threshold and yesterday's document is read by a different model today and may produce a different value. That is a change-control event, and treating it as a configuration tweak is how an accuracy regression arrives with no diff to point at.

So the policy belongs where the rest of the operational contract lives — in the control plane, versioned, per tenant, promoted through environments, with an owner and a rollback. And every processed document needs to record which policy decided its fate. The minimum record:

• The policy version — which routing configuration was in force at the moment of processing, so a result can be reproduced against the rules that produced it rather than today's rules.
• The signals observed — text layer present, capture statistics, layout indicators, sender history, latency class; the inputs to the decision, not just its output.
• The tier chosen and why — which rule matched, expressed in terms a person can read, because "the model picked it" is not an answer anyone accepts twice.
• Every escalation — which check failed, which tier ran next, and what changed in the result, so the ladder can be audited for tiers that never earn their place.
• Model and version per tier — the actual model identifier that served the request, because a provider updating a model behind a stable name is invisible otherwise.
• Cost and latency per attempt — attributed to the document, which is the only way the routing policy can be evaluated against the thing it exists to optimise.

That record is not bureaucracy. It is the same trace you need for any multi-step extraction, extended with the decision that chose the steps — and without it you cannot answer the two questions that always arrive: why did this document get a different answer than that one, and what would this document produce if we ran it again today. In a regulated workflow, the second question is the whole audit.

Four ways routing goes wrong

1. The router costs more than it saves

Every signal computed, every check run, every extra network hop and every queue is overhead paid on 100% of volume to change the handling of some fraction of it. When the fraction is small or the tier spread is narrow, the arithmetic does not close. The test is unsentimental: measure the fully-loaded cost of the routing path — including the checks, which are often the largest part — against the counterfactual of sending everything to one tier. Do it per document type, because the answer usually differs sharply between them, and be prepared to route only the segments where it wins.

2. Routing on price alone

A router that picks the cheapest endpoint meeting a latency target will silently absorb every accuracy difference between the endpoints, and those differences are not stable — the model behind a name changes, a provider adjusts defaults, a quantised variant appears under the same label. Without a per-tier regression set that runs on a schedule, the first signal is a rise in exceptions that nobody connects to a routing table. Price is a valid input; it is not a valid sole criterion, and what you can verify about a provider belongs in the same decision.

3. Thresholds tuned once, on a mix that no longer exists

Routing thresholds are fitted to an input distribution, and the distribution moves: a new channel opens, a large customer changes their template, a scanner is replaced, a supplier switches to photographs. The policy that was optimal in March quietly becomes a policy that sends readable pages to the expensive tier and hard pages to the cheap one. Routing decisions and their outcomes have to be reviewed against the current mix on a fixed cadence, and the review needs the record described above to be possible at all.

4. Fallback that hides the failure

The most damaging pattern is also the most reasonable-looking. The preferred tier times out or errors, the system falls back to a cheaper one, the cheaper one returns a result, and the result is emitted with no marking. Now a degraded read is indistinguishable from a normal one, and the incident that follows has no signature to search for. A fallback that crosses tiers must be recorded as a fallback, must re-run the same check set rather than a relaxed one, and — where the fields are consequential — should produce an exception rather than a value. Availability is a real requirement; satisfying it by lowering the standard without saying so is not a trade-off, it is a hidden one.

The trade-off, stated honestly

Two arguments against routing, both of which are sometimes right.

The first is simplicity, and it is stronger than it sounds. One tier for everything is reproducible, easy to reason about, easy to evaluate, and easy to explain to an auditor: same input, same path, same model. A routed pipeline has a combinatorial surface — every tier needs its own regression set, its own thresholds, its own failure handling, and the interactions between them need testing too. That is a permanent engineering cost, not a one-off build. Below a certain volume, or with a narrow enough intake, paying frontier prices for every page is cheaper than the team-months required to avoid it, and choosing that deliberately is a good decision rather than a lazy one.

The second is that routing is sometimes sold as intelligence when it is margin management. A platform that routes your documents to cheaper models while charging a flat per-page price has an incentive that is not aligned with your accuracy, and the routing decision is invisible to you by default. That does not make it illegitimate — the vendor is entitled to run its own infrastructure economically, and often the cheaper tier is genuinely sufficient. It does make two questions fair to ask in procurement: what does the router optimise for, and can you see, per document, which tier ran. If the answer to the second is no, then the accuracy you measured during evaluation is not a property you have been sold — it is a property of whatever the routing table happened to be that week.

Where to start

A short sequence, ordered by information gained per hour spent.

• Measure the easy share first — what percentage of your pages arrive born-digital with a coherent text layer and a simple layout; that number is the ceiling on what routing can save, and it is a one-afternoon measurement.
• Price the fully-loaded path, not the model — include rendering, storage, egress, the checks and the router itself, then compare against sending everything to one tier per document type.
• Build the checks before the ladder — a validation set strong enough to catch a cheap tier's failures is the prerequisite for every downgrade; without it, routing down is just guessing more cheaply.
• Start with two tiers, not five — one cheap path with a hard eligibility rule and one strong default, with escalation between them; a five-tier ladder is unevaluable long before it is optimal.
• Make abstention a supported output at every tier — if a tier cannot say it does not know, it cannot be routed to safely, whatever its benchmark score says.
• Record the decision from day one — policy version, signals, tier, escalations, model identifiers, cost; retrofitting this after the first accuracy dispute is significantly harder than adding it now.
• Put a ceiling on effort before you enable the loop — maximum escalations per document, maximum spend per document, and a defined terminal state that is an exception rather than a best guess.
• Re-run the comparison quarterly — model prices, capabilities and your own input mix all move; a routing policy is a standing decision, not a completed project.

Closing thought

Routing is the point where a document pipeline stops being a model and starts being a system. The interesting engineering is not in picking the strongest model — that part is a benchmark and a purchase order. It is in knowing, before spending anything, what kind of page you are holding; in having checks good enough that a cheap answer can be trusted or caught; and in making the machine's decision about how hard to try into something a person can read back a year later.

At Cogneris we treat effort as an explicit, recorded property of processing rather than an internal optimisation: validation that names what failed and where on the page, abstention recorded as a result rather than a guess, per-tenant configuration for how documents are handled, and tamper-evident audit metadata covering what was extracted, what was checked and what was released. If your unit costs are healthy and your exception rate is not — or the reverse — the routing policy is usually where the answer is. See how the API models these boundaries, the pricing page, or talk to our team. The cheapest read is the one you can prove you did not need to repeat.