Retrieve the passage, or extract the field.

A team loads three hundred supplier agreements into a vector store and wires a chat interface on top. The demo goes well. Someone asks what the termination clause in the Kestrel Supply agreement says, and the answer comes back with the right paragraph and a page reference. Then a finance lead asks which agreements renew automatically before 31 December, and the system returns five confident names. There are nineteen.

Nothing in the stack failed. The retriever did exactly what it was built to do: it ranked passages by similarity and handed the model the best few. The second question was simply not a retrieval question. It needed a value from every document, checked and comparable, and a ranked list of passages cannot promise that.

That is the decision this article is about. When documents go into a RAG system, some questions should be answered from chunks and some from fields. Most production systems need both, and the expensive mistakes come from using one where the other belongs.

Two questions that look alike

“What does the termination clause say?” and “Which agreements renew before December?” read like the same kind of request. Both are natural-language questions about the same corpus. Underneath, they ask for different things.

• A passage question — the answer is text a person will read, and wording, qualifiers, and surrounding context are part of the answer.
• A value question — the answer is a typed value, often across many documents, and software may sort, sum, filter, or act on it.
• A mixed question — the system has to find records by value and then explain one of them in words, such as why one renewal term differs from the standard template.

Retrieval is good at the first kind. It is ranked and truncated by design: the retriever returns the top k candidates, and anything below the cut does not reach the model. For a passage question that is fine, because one good passage answers it. For a value question it is a silent recall ceiling. Nineteen matching agreements and a k of eight can never produce a complete list, however good the embedding model is.

A decision rule you can apply per question

Classify the questions your users and workflows actually ask before choosing a representation. Four tests separate them quickly.

When a question answers yes to tests pointing both ways, it is a mixed question. Use the fields to decide which documents are in scope and the chunks to explain them. The failure mode to avoid is asking a language model to perform the first half by reading retrieved text, because that is where totals come out plausible and wrong.

The document AI for RAG page states the same split in operational terms: retrieval is measured by answer relevance and citation quality, extraction by field accuracy and review effort, and the failure modes differ accordingly — a wrong or ungrounded answer on one side, a wrong value posted to a system of record on the other.

Where generic chunking breaks

The default chunker in most tutorials splits extracted text every few hundred tokens with some overlap. On prose it works well enough. On business documents it breaks in predictable places, because business documents carry meaning in their layout.

• Tables — a rate table split mid-way puts the column headers in one chunk and the rows in the next. The retriever can return “2.5%” with no record of whether it is a discount, a late fee, or an index cap.
• Key-value forms — a label on the left margin and a value in a box can land on opposite sides of a boundary, or be interleaved with a neighbouring field in extraction order.
• Headers, footers, and footnotes — a repeated page header adds the same noise to every chunk and pulls unrelated chunks together, while the footnote that qualifies a figure ends up pages away from it.
• Headings — a chunk that begins in the middle of section 14.3 does not say it belongs to “Termination”. Similarity search then matches on vocabulary rather than on the section the text belongs to.
• Multi-column and scanned pages — if the text layer’s reading order is wrong, the chunker faithfully preserves the wrong order and the embedding encodes a sentence the document never contained.

None of these is fixed by a better embedding model. They are parsing problems that surface as retrieval problems. The practical remedy is to parse structure first and cut along it: keep a table as a unit, or repeat its header row in every piece when it must be split; keep a label with its value; carry the heading path with the text; and strip repeated page furniture before embedding.

What a chunk has to carry

A chunk that contains only text can support an answer but cannot defend one. The retriever cannot filter it, the reviewer cannot find its source, and the application cannot connect it to anything else it knows about the document. Store the context with the text.

• Identity — the document ID, document type, and the parse or extraction job that produced the chunk, so a re-parse replaces chunks rather than duplicating them.
• Location — page number and bounding box, so a citation opens the exact region instead of a whole file.
• Structure — the section heading path, and a table ID when the chunk comes from a table, so similar text in different sections stays distinguishable.
• Quality — a confidence or source-quality signal, so a low-quality scan can be re-ranked below a clean native PDF.
• Links to fields — the IDs of any validated fields extracted from the same region, so an answer quoting a value can be checked against the stored value.

Most of this list is what the RAG page describes as embedding metadata: page, section, document type, table ID, bounding box, citation, confidence, and job ID travelling with every chunk. Its purpose is not tidiness. It is what lets the retriever filter by document class and lets a reviewer trace a sentence back to the page it came from. The source citations page covers the reviewer’s side of that trace.

When the answer is a field

A field is a commitment the chunk never makes. It has a name, a type, a value, a validation state, and a citation to the region that supports it. “Renewal date: 2026-11-30, type date, validated, page 7” can be sorted, compared with today, and handed to a scheduler. The same fact inside a chunk is a sentence that a model has to find, read, and restate each time it is asked.

That is why value questions belong on fields:

1. Extract the fields the workflow depends on — counterparty, effective date, renewal date, notice period, auto-renewal flag — once per document.
2. Validate them at extraction time: a date parses, a notice period is a duration, an auto-renewal flag is a boolean and not the word “see clause 9”.
3. Send the values that fail validation to review, with the citation, before anything downstream depends on them.
4. Answer the question with a query over stored values, which is exhaustive by construction.

Fields also make retrieval better. Use them as metadata filters before similarity search runs: restrict to supplier agreements, to one counterparty, or to documents effective after a date, and then rank passages inside that set. Filtering first removes the near-duplicate templates that otherwise crowd the top of the ranking.

The cost is up-front definition. A field has to be named before it can be extracted, while a chunk index accepts whatever arrives. For questions nobody anticipated, retrieval is the right starting point. When the same value question keeps recurring, promote it to a field.

A worked example: one agreement set, three questions

Take the three hundred supplier agreements again, parsed once, with both outputs stored against the same document IDs.

The first path needs no fields. The second never calls the retriever. The third uses the field to establish a fact and the chunks to explain it, which keeps the model from inventing the difference it has been asked to explain. A router in front can classify questions, but the classification is only possible if both representations exist.

Parse once, then branch

Running a retrieval parser and a separate extraction pipeline over the same documents produces two readings of every page that can disagree about reading order, table boundaries, and even page numbers. Then a cited chunk and a validated field cannot be reconciled because they do not share coordinates.

The sturdier pattern is one parse with two projections. Layout, tables, headings, and page geometry are recovered once. Chunks for retrieval and fields for automation are derived from that single result and keep the same document ID, page numbers, and regions. The document ingestion contract article describes the same idea from the API side: one content graph, with search text, chunks, and typed values as views of it rather than competing parse results.

Three neighbouring decisions are easy to confuse with this one, and each has its own article:

• Splitting is not chunking — dividing a long packet into logical documents decides the unit of extraction, while chunking decides the unit of retrieval. Deep split for long documents covers the first.
• Every chunk is a copy — an embedding built from a document is a derived store that erasure has to reach. Machine unlearning for document AI covers the deletion side.
• Retrieved text is untrusted input — a passage pulled into context can carry instructions. Prompt injection in document AI covers why retrieved content needs its own trust boundary.

Test the two paths differently

One accuracy number for a RAG system hides the distinction this article draws. Build separate evaluation sets.

• Retrieval recall — for questions with a known source page, check whether that page appears in the top k, and track how often it does not.
• Citation correctness — check that the cited region actually contains the quoted text, not merely a page from the right document.
• Table integrity — ask questions whose answer is a single table cell and verify that the returned value comes with its column and row meaning.
• Field accuracy — compare extracted values with a labelled set, per field, and report the validation failure rate beside it.
• Completeness on value questions — for aggregate questions, compare the returned set with the full ground-truth set. This is where chunk-only systems fail quietly.

The last test is the one that exposes the opening example. It is cheap to write and it settles the architecture argument with a number rather than a demo.

Chunk for reading, extract for acting

Retrieval answers questions whose answer is a passage someone will read. Extraction answers questions whose answer is a value something will act on. Most real questions lean clearly one way, and the rest are a field lookup followed by an explanation.

Decide per question, not per project. Parse the document once, keep structure intact when you cut it, carry the context with every chunk, and promote recurring value questions to validated fields. The chat interface can stay the same. What changes is whether the nineteenth agreement shows up.