Summarising a 400-page PDF
In one line
The window is large enough to hold the whole document; putting it there is the expensive, lossy, unrepeatable option, and the alternative is a map-reduce you already know how to build.
The situation
A 400-page regulatory filing lands and someone needs the ten things that matter in it. The model's is advertised at 200k tokens; the document is about 300k. Even if it fit, the instinct to "just put the whole thing in" is worth resisting on three separate grounds — and the alternative is a pattern from ordinary distributed computing.
Why one giant call is the wrong shape#
Cost curves upward. compares every token with every other, so the work grows with the square of the sequence. Doubling the input more than doubles the cost, and the per-request memory grows with it.
The middle gets lost. Models attend more reliably to the start and end of a long prompt than to its middle. A fact on page 210 of 400 is the one most likely to be skipped, and you have no way of knowing whether it was.
It is one unrepeatable shot. If the summary misses something, the only option is to run the whole thing again with a different prompt, at the same cost, with no way to check which part went wrong.
Map, then reduce#
Split the document into sections — by the filing's own structure, not by page count, which is the same instinct as retrieval. Summarise each section independently, in parallel, with a prompt that asks for exactly the shape you want: obligations, dates, amounts, parties. Then run one more call over the section summaries to synthesise the top ten.
Cost is now linear in the document. The calls run concurrently, so wall-clock time is one section's worth, not the whole book. Every intermediate summary is inspectable — when the final result misses something, you can find the section that dropped it and fix that prompt. And the section prompt is identical across calls, so its instructions sit in a cached prefix and are prefilled once; makes the map phase cheaper than its token count suggests.
For a document this size a middle layer sometimes helps: summarise sections into chapters, chapters into the whole. The shape is a tree, and the reduce step at each level is small enough to read.
What gets lost is cross-section reasoning — a clause on page 40 that modifies one on page 300. If that matters, retrieval over the sections with a targeted question handles it far better than hoping a single 300k-token pass noticed.
Questions to ask
- Does the document have its own structure to split on?
- What shape should each partial summary have, so the reduce step has consistent inputs?
- Which step would I inspect first if the final answer looked wrong?
- Are there cross-references that need a targeted question rather than a summary?
Mindset
A large context window is a limit, not an instruction. When the input is bigger than a few thousand tokens, reach for the shape you would use for any large job: split it, process in parallel, combine — and keep every intermediate result where you can look at it.
Where it connects
Got the shape of it?
Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.