Beyond the Reranker

• Beyond the Reranker (arXiv 2606.28367)

This paper from Cascade Research asks if the many popular retrieval enhancements actually still help once you’ve added a strong cross-encoder reranker to a RAG pipeline? Most evidence for methods like query expansion, hierarchical summarization, graph expansion, routing, and rank fusion comes from homogeneous corpora (mostly Wikipedia prose), so the authors built HetDocQA, a benchmark whose collections mix code, markdown, prose, tables, and scientific PDFs. It uses character-span relevance labels matched to each system’s own chunks at evaluation time, which makes retrieval scores independent of how a system segments the corpus, and it splits data disjointly by collection so tuned thresholds can’t exploit recurring structure. They ran eight methods on a shared backbone (fixed embedder, reranker, and generator), paired HetDocQA with MuSiQue and QASPER as controls, and applied bootstrap confidence intervals with multiple-comparison correction. They find that the reranker carries almost all the retrieval quality, and beyond it only two methods give reliable gains: query expansion (HyDE), which helps when the question and evidence share little surface vocabulary, and SSCC, a per-source calibrated corrector that sets a separate acceptance threshold for each score source and helps only on heterogeneous data. Everything else that reranks or expands the candidate pool (RAPTOR, cross-document summarization, graph expansion, routing, rank fusion, corrective re-retrieval) shows no reliable improvement once the reranker is present, with the intuition being that a good reranker has already ordered a pool that contains the answer, leaving room to help only at the input (what enters the pool) and the answer decision (what gets accepted).