Documentation stopped being a deliverable and became a gate

There is a standard order of operations in enterprise software, and everyone reading this knows it. You build the thing. You sell the thing. Somewhere between the second and the fifth enterprise customer, a security questionnaire arrives, then a vendor risk review, then an auditor, and the documentation gets written in the shape of whatever was asked. The artifacts are real, they are just retrospective — a description of a system that already exists, produced by people reconstructing decisions from commit history and memory.

The high-risk regime inverts that order. The obligations that became enforceable on August 2, 2026 attach before a system is placed on the market or put into service: a technical documentation file describing the system, a risk-management process that has been running rather than promised, evidence about the data the system was built and tested on, stated accuracy and its limits, human oversight measures that exist in the product, logging that produces a record, a conformity assessment, a declaration of conformity and registration in a public database. None of that is new in content. What is new is that it is chronologically prior to deployment, which means a team that treats it as a follow-up item has not delayed a document — it has deployed a system it was not yet permitted to deploy.

There is calendar noise around this, and pretending otherwise would be dishonest: through 2026 there have been proposals to stagger or soften parts of the high-risk regime, and it is entirely possible that some specific dates move again. The strategic reading does not move with them. Every direction of travel — European, sectoral, and the procurement practices of large buyers who are not waiting for legislators — converges on the same demand, which is that a system making consequential decisions about people should be able to state what it does, how well, on what data, with what supervision, and produce evidence rather than assurances. A team that builds toward that is early under one calendar and merely prepared under another. A team that waits is betting on the delay.

The scope question is separate and comes first, and we have written about it at length in the piece on classification and the high-risk deadline: what puts a document pipeline in scope is the use it feeds, not the sophistication of the model. An extraction service reading invoices for a bookkeeping tool is not high-risk. The same extraction service reading the same invoices as an input to a creditworthiness decision inherits the classification of the decision it serves. This piece assumes that question has been answered and the answer was yes.

What the dossier actually has to establish

"Technical documentation" is a misleading name, because the word documentation suggests writing and the work is mostly measurement. Roughly eight things have to be established, and the useful way to read the list is as a column of claims that must be true, each paired with the artifact that makes it checkable by someone who does not trust you.

Section What it has to establish Where document teams come up short
System & purpose What the system is, the intended purpose stated narrowly, the versions in scope, and the reasonably foreseeable misuse it is not for. A product description written for buyers, which is broad on purpose and therefore claims a scope nobody can evidence.
Architecture & components The pipeline stages, the third-party models and services inside it, and what each contributes to an output. Model providers treated as an implementation detail rather than a documented dependency with a version.
Data governance Provenance, selection criteria, preparation and known limitations of training, validation and test data — including bias examination. Test sets assembled from whatever customer documents were handy, with no record of how they were chosen or what they represent.
Performance & limits Accuracy metrics, the conditions under which they hold, and the populations or document classes where they degrade. One platform-wide accuracy figure from a benchmark, unsegmented and untraceable to a fixed test set.
Risk management A running process: identified risks, mitigations, residual risk accepted by a named owner, revisited on a cadence. A risk register written once for a questionnaire, never updated after a model change.
Human oversight The measures built into the system so a competent person can understand, question, override or stop an output. A review queue that exists but has no thresholds, no authority rules, and no record that anyone actually looked.
Logging & traceability What the system records automatically, for how long, and how a past output is reconstructed from it. Application logs kept for 30 days for debugging, which is not the same artifact as a record of a decision.
Change & monitoring Versioning of the system, what counts as a substantial change, and how live performance is monitored after release. Continuous deployment with no notion of which release a given historical output came from.

Read the right-hand column again. Every entry is a normal engineering practice that a competent team skipped for a defensible reason — speed, scope, the absence of anyone asking. That is the point. The dossier is not hard because the requirements are exotic. It is hard because it asks for records of decisions that were made informally, and informal decisions leave no records.

Intended purpose is narrower than a product description

The first section is the one teams write fastest and get wrong most often, because there is an existing document that looks like it — the product page — and it was written to be expansive. A statement of intended purpose has the opposite job. It bounds the system: these document types, in these languages, at these quality levels, feeding this decision, for these users, with these known limitations. Everything outside the boundary is either out of scope or has to be evidenced too.

The practical test is uncomfortable and worth running early. Take your marketing copy, strike every capability you cannot produce a measurement for, and see what survives. If the page says the system handles handwritten forms and you have no handwriting test set, the choice is to measure it or to stop claiming it. This is the same discipline as refusing to describe a workflow as agentic when it is a sequence of scripted calls, applied to a document with legal weight instead of a landing page.

Accuracy is per document type, not per platform

A single accuracy number for a document AI platform is close to meaningless, and under a conformity assessment it is worse than meaningless because it invites a question you cannot answer. Extraction quality varies enormously by document type, layout family, scan quality, language and field. A system at 97% on clean digital invoices might sit at 78% on photographed receipts, and a payslip from one payroll provider is a different extraction problem from a payslip from another. The aggregate hides exactly the variation a regulator or a customer cares about, because the harm concentrates in the weak segment.

What the dossier needs is narrower and considerably more work: metrics stated per document type and per critical field, against a test set whose composition is documented and frozen, with the failure modes named rather than averaged away. That means field-level precision and recall rather than a document-level pass rate, the confidence distribution rather than a mean, and an explicit statement of the conditions under which the numbers stop holding. It also means the test set is a governed artifact — versioned, with a documented selection rationale, and separate from whatever data trains or tunes the system. Teams that have been tracking cost per successful task already have most of the measurement scaffolding for this, because both require knowing what "successful" means per document type instead of in aggregate.

Human oversight has to be a capability, not a sentence

The oversight requirement is where the gap between written compliance and built compliance is widest. It is easy to write "outputs are reviewed by a qualified operator." It is considerably harder to show that the person reviewing can actually understand what the system produced, has the information needed to question it, has the authority and the interface to override it, and is not so buried in volume that review has become rubber-stamping.

Built oversight has recognisable properties. There is a threshold policy that decides what escalates, defined per field and per document type rather than one platform-wide confidence cutoff, because the cost of being wrong on a total payable is not the cost of being wrong on a supplier's address line. The reviewer sees the source — the page region a value came from — rather than a bare field, which is what makes questioning possible at all. Override is a first-class action that is recorded with a reason, not a database edit. And someone is measuring the override rate, because a review step where the human agrees 100% of the time is a review step that has stopped functioning and should be detected, not discovered. This is the operational half of validating at capture time rather than downstream: oversight that catches problems while a person is still in a position to act on them.

A compliance document that describes a system you did not build is not evidence. It is a statement of intent with a version number on it, and the gap between the two is exactly what an assessment is designed to find.

Risk management is a live system, not a document

The risk-management requirement is written as a continuous process, and the word doing the work is continuous. Identify the risks the system poses in its intended use and under foreseeable misuse, mitigate what can be mitigated by design, control what cannot, name the residual risk and have someone accountable accept it, then revisit the whole thing when the system changes or the world does. A register produced once and filed is not compliance with that requirement — it is documentation of a moment.

For document workflows the risk inventory has a fairly consistent shape once you sit down to write it: extraction error propagating into a decision, systematic degradation on a document class associated with a particular population, over-reliance by reviewers on a system that is usually right, instruction injection through document content, forged or synthetic source documents, silent behaviour change after a model update, and unavailability during a period where decisions still have to be made. Each needs a mitigation and a residual statement. The one that most often goes unwritten is the second — degradation concentrated in a segment — because finding it requires segmented measurement, which loops back to the accuracy section. The sections are not independent, and that is why assembling them in parallel at the end does not work.

Logging is what turns a claim into evidence

Automatic record-keeping is the requirement that quietly determines whether any of the others can be demonstrated a year later. The dossier states what the system records; the records are what prove the system behaved as the dossier says. Debug logs are not this. A record that supports an assessment has to let someone reconstruct a specific past output: which input, which system version and model version, which configuration and thresholds were in force, what the system produced, what a reviewer did to it, and when. Retention has to outlast the decision's consequences, which for a credit or eligibility decision is measured in years rather than days.

The half most often missing is the policy half. Most platforms log what happened to a document. Far fewer log what the system was permitted to do at that moment — the thresholds, the routing, the model pin in force — which is why the answer to "why did this auto-release" so often requires reading code and commit history. That is the argument for holding policy in a control plane and logging the resolved policy alongside each job, and it is the difference between logging outputs and holding an evidentiary record of a non-deterministic system.

Why this takes months from a standing start

The estimate teams give themselves is a sprint, because the output is a document and documents are written quickly. The estimate that holds is a quarter or more, and the reason is that four of the eight sections depend on data that has to be produced before it can be written down — and none of it can be produced retroactively without lying.

The measurements do not exist at the required granularity. Building a per-document-type, per-field accuracy report means defining document types formally, assembling and labelling a representative test set for each, deciding what counts as a correct field, and running the evaluation. Labelling alone is weeks of work with real disagreement about edge cases. There is no shortcut: an unmeasured claim is the thing the section exists to prevent.

Data lineage was never recorded. Describing the provenance and selection of training and test data requires knowing where each set came from, on what basis it was chosen, and what rights attach to it. If the test set was accumulated informally from customer files, the reconstruction is part archaeology and part uncomfortable legal conversation. Worse, some of it may be unusable once examined, which means rebuilding rather than documenting.

Oversight was designed for throughput, not for evidence. If the review interface does not show the reviewer where a value came from, or override is not distinguishable from a normal edit, the fix is a product change on a roadmap, not a paragraph. Teams routinely discover this three weeks before an assessment date.

Nothing was versioned that needed to be. Continuous deployment without release identity means an output from March cannot be attributed to a system state. Retrofitting version identity is straightforward going forward and impossible backwards, which puts a hard floor under how much history can ever be covered. This is the same dependency structure as maintaining an AI bill of materials — cheap to keep, expensive to reconstruct, and never fully recoverable for the period before you started.

The compounding effect is what makes the schedule slip. The accuracy section needs the test sets that the data section is still assembling. The risk section needs the segmented results the accuracy section has not produced. The oversight section needs a product change to be shipped before it can be described truthfully. Sections that look parallel on a plan are serial in practice, which is the ordinary reason pilots stall before production in regulated contexts.

What a buyer should demand at the proof of concept

If you are deploying someone else's document AI inside a high-risk workflow, their dossier is load-bearing for yours. The moment to establish that is the proof of concept, when there is still leverage — not after signature, when the answer to every awkward question becomes "it's on the roadmap." Six artifacts are worth asking for by name, and the request is reasonable enough that a vendor's reaction to it is itself information.

  • Segmented accuracy with the test set defined — metrics per document type and per critical field, plus how the test set was composed. A single headline number and no test-set description means the measurement infrastructure does not exist.
  • A written statement of intended purpose and limitations — including document types and conditions where performance degrades. A vendor unwilling to write down limitations has not measured them.
  • The record specification — what is logged automatically, for how long, in what form it can be exported, and whether it includes the configuration in force, not only the output.
  • Version pinning and change notification — whether you can pin a model and schema version, how substantial changes are communicated and with what notice, and whether you can decline or defer one.
  • The oversight design — what escalates and at which thresholds, whether thresholds are configurable per field, what a reviewer sees, and how overrides are recorded.
  • The responsibility split in writing — which obligations the vendor carries as provider, which fall to you, and what they commit to supply when you are assessed. This belongs in the contract, not in a slide.

The list overlaps heavily with what a management-system certification would have a vendor demonstrate anyway, which is why an ISO 42001 posture has become a procurement baseline rather than a differentiator. It is not a substitute for the dossier — a management system says you have a process, the dossier says what this system does — but a vendor with the former usually has the raw material for the latter, and a vendor with neither is asking you to carry their evidence gap on your own assessment.

Where the provider and deployer split runs

The obligations divide between the organisation that develops and places the system on the market and the organisation that uses it under its own authority, and the division is less convenient than most buyers assume. The provider builds and maintains the technical documentation, runs the conformity assessment, registers the system and monitors it after release. The deployer uses it in accordance with the instructions, makes sure input data is relevant to the intended purpose, assigns human oversight to people with the competence and authority to exercise it, monitors operation, keeps the logs it controls, and informs affected people where required.

Two edges are worth stating plainly, because they are where teams get surprised. First, the deployer's obligations are not paperwork — "input data is relevant to the intended purpose" means that feeding a system document types outside what the provider evidenced is your risk, and "competent oversight" means the reviewer has to be trained and empowered, not merely present. Second, the roles are not fixed by contract. Putting your own name on a high-risk system, or modifying it substantially, or using it for a purpose the provider did not declare, can move you into the provider's obligations for that system — with the full documentation burden attached. Teams that wrap a vendor extraction API in their own decision logic and their own brand should read that sentence twice, and should hold the boundary explicitly the way an agent registry holds ownership and review dates: named owner, declared purpose, documented scope, reviewed on a cadence.

The overhead, stated honestly

This is real cost, and the case for it is weaker than compliance marketing suggests in at least two directions.

It buys nothing outside scope. A dossier for a workflow that is not high-risk is expensive theatre, and a meaningful number of document pipelines are genuinely not in scope — extraction feeding bookkeeping, archival, search, internal reporting. Building the full apparatus for those is governance out of proportion to the risk, and it has a second-order cost: it trains an organisation to treat compliance as ritual, which is precisely the posture that fails when a genuinely high-risk use arrives. The scope question deserves a careful answer before the documentation question gets asked at all.

And the artifact decays. A dossier describes a system at a version. A model update, a schema change, a new document type, a new customer segment — each potentially invalidates a section, and the requirement is for documentation that stays accurate, not documentation that was accurate once. That converts a one-time project into a maintenance obligation, and it is the part that gets underestimated in every plan we have seen. The mitigation is structural rather than heroic: generate what can be generated. Accuracy tables that come from the evaluation pipeline, version tables from the model registry, data descriptions from the test set catalogue, change history from the policy log. The sections that are genuinely prose — intended purpose, risk analysis, oversight design — change slowly. The sections that change fast are the ones that can be emitted by a system rather than typed by a person, which is the same argument as monitoring controls continuously instead of sampling them annually.

The honest counterweight is that most of the underlying work is engineering you benefit from regardless of any regulator. Segmented accuracy tells you where the pipeline actually fails. A frozen test set makes model upgrades a comparison instead of a hope. Version identity makes incidents diagnosable. Threshold policy per field is how the review queue stops growing. Records that reconstruct a decision are how a customer dispute ends in twenty minutes instead of a week. The dossier is largely a report on whether you did those things — which is why teams that did them find the assembly tedious, and teams that did not find it existential.

Where we deliberately stop

We are a document AI platform, not a conformity assessment body, and the boundary matters. We do not certify anyone's system, we do not classify anyone's use case for them, and a vendor telling you your deployment is compliant is telling you something they are not in a position to know — classification depends on the decision your workflow feeds, which lives on your side. What a platform can do is make the evidence producible rather than reconstructable: extraction and classification configured per tenant and per document type, versioned output contracts, per-field confidence available to a threshold policy, tamper-evident audit metadata, and contract-specific retention — so that the accuracy, oversight, logging and change-control sections of your documentation describe a system that behaves that way, rather than one that was described that way. Everything above the platform line — your intended purpose, your risk acceptance, your oversight staffing, your assessment — remains yours, and any vendor claiming otherwise is selling you a comfort that will not survive contact with an assessor.

Closing thought

The instinct that the dossier is a writing task is what makes it a crisis. It is a measurement task with a written summary attached, and the measurements take a quarter to produce, in a serial order, from data most pipelines were never instrumented to emit. The teams that will find August 2026 uneventful are not the ones with the best compliance counsel. They are the ones who versioned their models, froze their test sets, segmented their accuracy, made override a recorded action and kept records that reconstruct a decision — for engineering reasons, before anyone asked. The dossier, for them, is a report. For everyone else it is a project that has to finish before the thing they already built is allowed to run.

At Cogneris we build document AI on the assumption that the evidence will eventually be asked for: per-tenant and per-document-type configuration, versioned output contracts, confidence exposed per field so thresholds are yours to set, audit metadata designed to be exported rather than screenshotted, and retention that follows the contract rather than a storage default. If you are standing up a document workflow that feeds a consequential decision and the documentation question has moved ahead of the go-live date, see how the API models these boundaries, read what we hold on the security and evidence side, or talk to our team. A system that cannot produce its own evidence is a system you will end up documenting by hand, every year, forever.