Retrieval over company data

How does an assistant find the right passage?

The question is answered before the model sees it. The corpus is split into passages, each passage is turned into a vector, and the vectors are stored in an index. At question time the question is turned into a vector too, the index returns the nearest passages, and those passages are placed in the context window alongside the question. The model reads what it was handed.

That order matters, because it means the model is not searching. Retrieval decides what the model can possibly say, and the model decides how to say it. A confident, well-written, wrong answer is usually a retrieval fault wearing a generation costume.

The index inherits every filing decision

An index is built from a permission boundary. Whatever the connector was allowed to read, the index holds. If the connector ran with a broad grant, every document it could open is now one question away from anybody the assistant serves.

This is where the assistant gets blamed for something it did not do. The exposure was created by years of sharing decisions, and retrieval made it reachable in a sentence.

What makes retrieval fail?

Four things, in our experience, and they have different fixes.

Failure	What it looks like	Where the fix sits
Nothing relevant retrieved	A confident answer built from unrelated passages	Chunking and the embedding model
The right passage ranked too low	A partial answer that omits the governing clause	Reranking, and how many passages are passed
Stale content retrieved	An answer that was correct two policy versions ago	Index freshness and dated editions
Content retrieved that should not be	A correct answer the asker should not have seen	Permissions carried into the index

Only the last is a security problem. The first three are quality problems, and they are measured the same way answer quality is measured: a graded set of real questions with reviewed expected outcomes, scored on a schedule.

What to measure

Retrieval has its own error rate, separate from the answer’s. For each question in the graded set, record whether the passage that contains the answer was retrieved at all, and where it ranked. A system that retrieves the right passage nine times in ten and answers well eight times in ten has a generation problem. One that retrieves it six times in ten has a retrieval problem, and no amount of prompt work will fix it.

Measure the two separately or the number tells you nothing about what to change.

What do we do about it?

We treat the index as part of the permission boundary rather than as a search feature. Before an assistant is connected, the data risk assessment establishes what the connector can reach and what sits inside it. Where a clean-up is unfinished, the assistant is scoped to the content that has been reviewed, and widened as the permissions work lands.

The graded set covers retrieval and answer separately, and it runs weekly, because a corpus changes underneath a system that was correct when it launched.