Chunking a contract so the answers survive
In one line
A 500-token window cut clause 7.2 in half; the document already had boundaries, and using the author's beats any character count.
The situation
A legal team wants to ask questions of their master agreements. The first build chunks each contract into fixed 500-token windows, embeds them, and retrieves the top three per question. Asked about notice periods for termination, it returns half of clause 7.2 — the half that does not contain the number — and a paragraph from clause 12 that mentions "notice" in a different sense.
The retrieval is working. The chunks are wrong, and they were wrong before the first question was ever asked.
Boundaries decide what can be found#
A chunk is the unit of retrieval. If the fact you need spans two chunks, neither chunk contains the fact, and no amount of search quality recovers it. is therefore not a preprocessing detail; it is the decision that sets the ceiling on everything downstream.
Fixed-size windows ignore the document's structure and split at arbitrary points — mid-sentence, mid-clause, mid-table. patches the worst of it by repeating a slice at each boundary, so a sentence straddling a break exists whole in at least one chunk. It helps, and it is still a patch.
Contracts are among the most structured documents that exist. They have numbered clauses, nested sections, defined terms, and headings that say what each part is for. Every one of those is a boundary the author drew deliberately.
Use the structure, then add context#
Split on clauses. Clause 7.2 becomes one chunk, whole, however long or short it is. Very long clauses split on their sub-clauses; very short ones can be grouped with their siblings. The chunk sizes vary, and that is fine — the does not care, and the budget is managed at retrieval time by how many chunks you send.
Then prefix each chunk with where it came from: "Master Services Agreement › 7. Termination › 7.2 Notice periods", followed by the clause text. That line costs a dozen tokens and does two things. The embedding now carries the clause's purpose, so "notice periods for termination" lands on it even if the clause itself never uses the word "termination". And the model, reading the chunk, knows what document and section it is looking at, which matters when defined terms and cross-references are involved.
Cross-references are the remaining hard case: clause 7.2 says "subject to clause 15.3". A chunk cannot contain what it points to, but it can be retrieved alongside it. A cheap pass after that pulls in any clause a retrieved chunk explicitly references is usually enough.
Questions to ask
- What structure does this document already have, and am I using it?
- What is the smallest unit a question is likely to be answered by?
- Does each chunk know where it came from, when read on its own?
- Which facts span boundaries, and what happens to them?
Mindset
Chunk boundaries set the ceiling on retrieval before any query exists. Look for the boundaries the author already drew, keep each chunk self-describing, and treat "fixed 500 tokens" as the fallback for documents with no structure — which is rarer than it seems.
Where it connects
Phase 1 · 6 min
Embeddings
Text compressed into a fixed-length vector where geometric distance stands in for similarity of meaning.
Phase 3 · 7 min
Retrieval & RAG
Fetch the relevant text at request time and put it in the prompt, because almost everything worth answering about is either private, recent, or needs a citation.
Got the shape of it?
Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.