Full enumeration instead of top-k: map-reduce in research mode

Normal chat and research mode search the same knowledge base — but in fundamentally different ways. This article explains the difference and why it is decisive for certain questions.

The problem with top-k

For a normal question, CompanyWizard retrieves the most relevant passages and passes that handful to the model as context (RAG). How many passages that is, is capped — the k in “top-k”. → RAG & search

For most questions that’s exactly right: you want an answer, not a full-text review, and the best-matching documents contain it.

For enumerating and aggregating questions, though, the principle breaks down. If you ask “Which customers have a data processing agreement?”, the correct answer is the complete list. Top-k gives you the customers from the most relevant documents — and says nothing about all the others. The result looks complete but isn’t. The same goes for “How many projects use technology X?”: only what fell inside the slice would be counted.

The problem isn’t that the search is bad. It’s built for a different job.

The answer: look at every document once

Research mode inverts the approach. Instead of asking “which documents match best?”, it sifts through all of them that the asker is allowed to see. The run does this in five phases, which you can also follow on the composer toggle while the research is running:

1. Planning. A single model call turns your question into concrete search terms. That is the only call made before any document is touched at all.

2. Selecting — complete, but without a model. Now the run goes through every accessible document and checks it against the planned terms. Crucially, this sifting happens purely locally. No document leaves the process, no model is asked. This is exactly where completeness comes from — nothing is skipped for having ranked too low in a similarity list.

3. PII check — of the selection only. The passages the selection kept are pseudonymized. Only those, because only those are about to be seen by the model.

4. Evaluating (map) — one call per document. The selected passages go to the model grouped by document: “What does this say about my question?” All matching passages of one document share one call; several documents are evaluated in parallel. The source line is attached to each finding by the code, not the model — so the report stays linkable without depending on the model’s diligence.

5. Summarizing (reduce) — in stages. The many small evaluations are pulled together hierarchically: groups into intermediate results, those into larger ones, until a single coherent report remains. This hierarchy is necessary because the sum of all individual findings would not fit into a single model call.

The result is a Markdown report based on a complete review rather than a slice. While it is being put together and delivered, the composer toggle briefly shows Composing report.

Why the source appendix does not come from the model

The report ends with a list “Cited sources (N)” — one entry per document a finding came from. That list is produced by the code, not the language model, and it is appended after the condensation.

The reason lies in phase 5. Condensation pulls many individual findings into one readable text, and a finding can fall by the wayside in the process — the model decides what belongs in the running text. If the source list came from the same hand, the source would vanish along with the finding, and the report would look complete without being so. That is precisely the kind of error that costs most on an enumerating question.

So every finding carries its citation along from phase 4, in a fixed position, and the appendix is assembled from those. Condensation may shorten — it can no longer swallow a proven source. For the same reason the appendix appears in the report even when the condensation fails outright.

The coverage header at the top of the report (documents searched, matches, passages evaluated, search terms) likewise comes from the code, not the model. It describes what the run did — not what the model says about it.

What it costs

Completeness has a price, and that price is the honest reason the mode isn’t always on:

• Time: the run takes minutes instead of seconds — usually a few, with many matches considerably more. It makes one model call per matched document, and the condensation can only run once the evaluation is in.
• Tokens: considerably more than a normal answer. But consumption follows the number of matches for your question, not the size of the knowledge base: sifting all documents is free because it needs no model, and only what the selection kept goes to the model. A narrowly framed question over a large knowledge base can therefore be cheaper than a broad one over a small base.
• One at a time: each user only ever has one research run started from the chat — a running web research blocks a second one too. Switch the toggle on in the meantime and you get a normal answer with a notice saying so. → Research mode

That’s why research mode is a deliberately switchable per-question mode and not a new default.

Robust against individual failures

With many evaluation calls, some will always fail — a timeout, an unexpected model response. In that case the run degrades instead of aborting entirely: the affected document drops out of the evaluation, and the report is still produced and delivered — with the note “N document(s) could not be evaluated” right below the header. A single failure therefore costs you sources, not the whole research run.

The same applies to a model that does not answer at all: every evaluation call has a deadline of 60 seconds, every summarizing call one of 180 seconds. If the provider stalls, the run loses the affected document or delivers the findings unsummarized — instead of waiting silently until the run-time limit.

There is one exception: if the PII scan fails, the run aborts rather than continue — see below.

Permissions and personal data

The run only sees what you may see. Enumeration is bound to your permissions; vaults and documents you have no access to do not exist as far as the run is concerned. → RBAC & confidentiality

For personal data the same principle applies as in the rest of chat: only pseudonyms cross the boundary to the language model, never the real values. The flow above is built precisely around that — there is exactly one point at which text leaves the process: the evaluation. The selection before it does read plain text but never hands it out, and the condensation after it sees model output only.

The report stays pseudonymized for the entire run and is only translated back into real values at run end. If the pseudonymization service is unreachable, the run aborts rather than hand unchecked plain text to the model (fail-closed). The run checks availability itself — before and after the scan phase — instead of relying on an outage being passed through as “nothing found”. → The PII shield

If you switch the shield off before starting

The above describes the normal case: shield on. If you switch the PII shield off and then start a research run, the entire run goes unpseudonymized — the passages reach the model as plain text. This is not an oversight but the same deliberate opt-out as in normal chat, only lasting minutes instead of a single message.

Two peculiarities worth knowing:

• What counts is the shield’s state at the start. The research run takes it from the message that triggers it and keeps it until the end. Switching the shield back on mid-run does not apply to the report already in flight.
• The shield costs you hardly any result quality. Evaluation works about as reliably on pseudonyms as on real names — the findings come out practically the same. Switching the shield off for a research run therefore buys little and gives away a lot.

→ The PII shield

Related

• Research mode — the feature from the user’s perspective
• Asking an enumerating question
• RAG & search · The PII shield
• Tools: research