hello·ai
ExampleBest after Phase 34 min

The bot quotes last year's policy, with a citation

#retrieval#debugging#observability

In one line

Retrieval worked exactly as built — it found the most similar document, which was the stale one — and the fix is in what gets indexed and logged, not in the prompt.

The situation

A customer asks the assistant about the cancellation window. It answers "30 days", cites cancellation-policy.md, and is wrong: the window changed to 14 days in January. The engineer checks the prompt, which clearly says to answer from the provided documents. The assistant did exactly that. The document it was provided was the old one.

This is the single most common failure in production, and it is worth understanding as a retrieval failure rather than a model one — because every instinct points at the model.

questionretrievepolicy-2024.md · 0.71policy-2025.md · 0.69modelboth versions live in the index; the old one wins by a hair; the answer is fluent, cited, and wrongnothing errored — the fix is a date filter on retrieval and a log of what was retrieved
Both versions live in the index and the old one wins by a hair. Nothing errored; the answer is fluent, cited, and wrong.

What actually happened#

When the policy changed, the new page was added to the knowledge base. The old page was never removed — it was archived, renamed, or left in a folder the ingest job still reads. Both versions were chunked and embedded. At query time the step returned the five most similar chunks, and the old version, being slightly closer in wording to the question, edged out the new one. The model was handed a stale passage and did its job faithfully.

Three things make this hard to catch. It is silent: no error, no empty result, a normal-looking citation. It is intermittent: on a differently phrased question the new chunk wins, so the bug does not reproduce reliably. And it looks like a , so the investigation starts at the wrong end of the pipeline.

The fixes are all upstream of the model#

Log what was retrieved. Every request should record the chunk ids and scores that went into the prompt. With that one field, this bug takes five minutes to diagnose instead of a week. Without it, every retrieval failure looks like a model failure.

Make the index reflect the truth. Ingest should delete or supersede, not only add. Each chunk carries metadata — source, version, effective date — and retrieval filters on it: current documents only, or the latest version per source. Superseded content should not be retrievable at all for a "what is the policy" question.

Measure on a schedule. Take a hundred real questions with known correct sources and check, nightly, that the right chunk is in the top five. This catches the next stale document before a customer does, and it catches the slower version of the same problem — an index that degrades as the corpus grows.

Show the source's date. If the citation says "cancellation-policy.md, last updated March 2024", a human reader has a chance of noticing. Surfacing provenance costs nothing and catches what the pipeline misses.

Questions to ask

  • Can I see, for any given answer, exactly which chunks were in the prompt?
  • When a document changes, what happens to the old chunks?
  • Does retrieval filter on version or date, or only on similarity?
  • Is retrieval recall measured, or only end-to-end answer quality?

Mindset

When a RAG system answers wrongly, check what was retrieved before touching anything else. The model can only be as good as the passages it was handed — and a stale passage, faithfully used, produces a confident, cited, wrong answer.

Where it connects

Got the shape of it?

Examples do not count toward phase progress — that stays on the topics. This is just so the list remembers what you have seen.