Coordinates are part of the contract.

A coordinate is a claim, not a decoration

Most extraction APIs treat geometry as a bonus field. The structured output is the product; the box is what the demo highlights in yellow. That framing survives right up to the first audit, the first reviewer disagreement and the first model upgrade, and then it fails in the same way every time: someone has to prove that a number in a JSON payload came from a specific mark on a specific page, and the only evidence is a rectangle nobody wrote a specification for.

A bounding box asserts something falsifiable. It says the value 1,284.00 was read from this region of page 3, in this coordinate space, at this rotation. That claim can be rendered, checked by a human, compared against a previous run and replayed a year later. Nothing else in an extraction response can do that. Confidence scores describe the model's state of mind; coordinates describe the document.

So the question is not whether to return geometry. It is whether the geometry is a contract — written down, versioned, tested — or an implementation detail that changes when someone swaps an OCR engine. This matters more as extraction gets used to make decisions rather than to fill forms, which is the argument behind source citations and behind an audit trail that identifies the exact configuration behind one extraction. A citation without a coordinate is a sentence about a document. A citation with one is a pointer into it.

Three APIs, three coordinate systems

This is not a hypothetical divergence. Read the published references for the three cloud document APIs most teams evaluate, and you get three conventions that agree on almost nothing except the origin being at the top left.

The fourth column is where integrations break. Textract's bounding box reference states the rule plainly: top and left are ratios of the overall document page size, so a 700 × 200 image with a box starting at 350 × 50 pixels returns 0.5 and 0.25. Multiply those by the wrong page dimension and every region lands in the wrong place, consistently enough to look deliberate. Google's response guide notes that when the API detects a coordinate value of 0, that coordinate is omitted from the response — a parser that reads positionally, or that assumes four keys per vertex, will silently misread every region that touches a page edge. Azure's layout model puts the unit on the page rather than on the geometry, which is the safest of the three designs and the easiest one to drop when a consumer only stores the polygon. Mistral's OCR 4 announcement is explicit that boxes localize text for in-context highlighting and reliable data pipelines, and that each block is localized, typed and scored.

None of these designs is wrong. They are simply four contracts, and any system that normalizes across more than one of them has already written a fifth. That fifth contract is usually undocumented, which is the actual problem.

Write the schema before you write the parser

The normalization layer exists whether or not anyone designed it. Give it a schema, and the assumptions become reviewable. A region record should be self-describing enough that a consumer who has never seen the upstream vendor can render it correctly.

{
  "region_id": "r_0142",
  "contract": "cogneris.regions/2",
  "page": 3,
  "page_index_base": 1,
  "page_size": { "width": 2480, "height": 3508, "unit": "px" },
  "space": "page-normalized",
  "origin": "top-left",
  "axes": { "x": "right", "y": "down" },
  "box": { "x": 0.1210, "y": 0.4432, "w": 0.3180, "h": 0.0214 },
  "polygon": [[0.121,0.443],[0.439,0.443],[0.439,0.465],[0.121,0.465]],
  "page_rotation_deg": 90,
  "region_rotation_deg": 0,
  "coordinates_apply_to": "post-rotation",
  "binds": { "field": "invoice_total", "char_span": [8412, 8421] },
  "provenance": { "engine": "ocr-e", "engine_version": "4.2.1", "page_render_dpi": 300 }
}

Every key above exists because its absence has caused a production defect somewhere. The ones that carry the most weight are these.

• An explicit space and origin — page-normalized, page-pixels or page-points named in the record, not inferred from whether the numbers look smaller than 1. A pixel coordinate on a small thumbnail can fall below 1 and pass that test.
• The page size the coordinates were computed against — a normalized box is meaningless without it the moment anyone wants pixels back, and the render that produced it may not be the render the consumer has.
• The page index base — 0-based and 1-based indexing coexist across vendors and internal tools, and an off-by-one page reference is invisible in aggregate metrics and obvious to the one reviewer looking at the wrong page.
• Both a box and a polygon — axis-aligned rectangles are what user interfaces draw; polygons are what survive skew and rotation. Storing only the rectangle discards information that cannot be recovered.
• The binding to the extracted value — a region that is not tied to a field and a character span is a picture, not evidence. The span is what lets a reviewer jump from a disputed number to the pixels behind it.
• Provenance for the geometry itself — engine, version and render resolution. Coordinates produced by two OCR engines are not interchangeable even when both are correct.

Rotation is where the contract breaks

Rotation is the single most reliable source of geometry defects, because there are at least three rotations in play and they compose. There is the rotation the page declares, the rotation a renderer applied before OCR ran, and the rotation of an individual region on an otherwise upright page — a vertical label on a form, a stamp, a sideways table header. Textract exposes the last of these as a discrete RotationAngle per word. Azure exposes the first as an angle on the page. Combining outputs means deciding, in writing, what the numbers mean.

1. Declare the reference frame — do the coordinates describe the page as stored, or the page as displayed after rotation is applied? Both are defensible; only one can be true in a given payload.
2. Keep page rotation and region rotation as separate fields — folding them into one number loses the ability to re-render, and the sum is not always what a viewer needs.
3. State the order of operations — rotate then normalize, or normalize then rotate. These produce different numbers whenever the page is not square, which is nearly always.
4. Carry the page dimensions from the same frame — a 90-degree rotation swaps width and height, and a record that rotates the box but not the page size is internally inconsistent.
5. Test the composition, not the parts — a fixture set with an upright page, a 90-degree page, a 180-degree page and an upright page carrying a rotated region catches the failures that four separate unit tests miss.

The reason to be pedantic here is that rotation defects do not look like defects. The extraction is correct, the confidence is high, the value lands in the right field, and only the highlight is wrong — until a reviewer trusts the highlight over the value, or a regulator asks to see where a number came from.

A page is not an image

The word "page" is doing too much work in most extraction payloads. A PDF page is a vector surface with a declared size in points. The thing an OCR engine actually looked at is a raster rendered from it at some resolution. A multi-page TIFF has images that are pages by convention. Azure's layout documentation goes further and defines page units per format: each image is one page unit, each PDF page is one page unit, and for office formats up to 3,000 characters counts as a page unit, with each worksheet or slide counting as one. Under that rule, "page 4" of a spreadsheet is a worksheet, not anything a reader would recognize as a page.

Three consequences follow, and all three are cheap to handle at design time and expensive later.

• Pin the render, not just the source — record the DPI or scale factor used to rasterize. Two runs of the same PDF at 200 and 300 DPI produce different pixel coordinates and identical normalized ones, which is a strong argument for storing normalized values as the canonical form.
• Separate the document page number from the processing index — a request that processes a page range, splits a batch or skips blank pages produces an internal sequence that no longer matches the document a human opens.
• Treat non-paginated formats explicitly — a spreadsheet or a slide deck needs a rule for what a region means before anyone tries to draw one on it, not after a customer asks why the highlight is missing.

These are the same boundaries that make table extraction hard to evaluate, because a table cell's identity is partly structural and partly spatial, and the two can disagree on a page that a renderer laid out differently from the way the original document described it.

The coordinate convention is an API version

Field names are versioned because a rename breaks a build. Coordinate conventions are usually not versioned, because a change breaks nothing at compile time and everything at render time. That asymmetry is why geometry regressions survive release processes that catch far less serious changes.

Version the convention, not just the payload. A contract string in every region record — the field in the schema above — lets a consumer refuse to render geometry it does not understand, instead of drawing it in the wrong place. Refusing is the correct behaviour; a rectangle drawn under a wrong assumption is worse than no rectangle, because it looks like evidence.

Validate visually and numerically

Geometry needs both kinds of proof, and neither substitutes for the other. Numeric checks scale and catch drift; visual checks catch the class of error where the numbers are internally consistent and describe the wrong thing.

• A locked fixture set with human-verified regions — a few dozen pages covering upright, rotated, skewed, multi-column, scanned and born-digital documents, each with its expected regions stored alongside the coordinate contract they were recorded under.
• Intersection over union with a declared tolerance — a single scalar per region, with different thresholds for different consumers. A highlight that guides the eye can tolerate loose overlap; a redaction mask cannot.
• An invariant suite that needs no ground truth — boxes inside page bounds, polygons non-degenerate and consistently wound, child regions contained by their parents, page numbers within the document's page count. These run on every document in production, not only on fixtures.
• A rendered overlay committed as a build artifact — draw the regions onto the page image and keep the result. A reviewer spots a mirrored page in one second; a metric may never flag it.
• Round-trip conversion tests — normalized to pixels and back, upright to rotated and back, vendor space to internal space and back. Every conversion in the system should be reversible within a stated epsilon.

Run all of it whenever the OCR engine, the renderer, the vendor API version or the normalization code changes — the same discipline a benchmark that separates parsing from reasoning applies to accuracy. Coordinate quality is a parsing property. It degrades independently of whether the extracted values are right, and an evaluation that scores only field accuracy will report a clean run while every highlight is off by a page.

What to require, and what to publish

Whether the geometry comes from a vendor or from your own pipeline, the same six questions decide whether it can be trusted. Ask for evidence, not for a data model diagram.

The reason to hold this line is not tidiness. Extraction output is increasingly the input to a decision — a payment, a claim, a credit file — and the defence of that decision is the ability to point at the document and show the mark. That is what a bounding box is for. Treated as metadata, it is a highlight that usually works. Treated as a contract, with a schema, a version, a rotation rule and a test that renders it, it is the part of the response that makes everything else checkable.