Reranker (Cross-Encoder)
Properties
created
22.03.2026, 19:00
modified
01.08.2026, 11:09
published
Empty
topics
Cross-Encoder, Reranking, RAG
authors
Jakub
ai-assisted
Yes
- A cross-encoder is a model that takes a query–document pair as a single concatenated input and outputs a scalar relevance score
- Contrast with a bi-encoder, which encodes query and document independently
# Topics
- Reranker Models
- Cross-Encoder Training
- Sentence Trasformers
# Architecture
- Input:
[CLS] query [SEP] document [SEP]— both texts concatenated and passed through the encoder together - Backbone: transformer encoder (BERT, RoBERTa, MiniLM)
- Head: linear layer on the
[CLS]token → scalar logit (or softmax for classification) - The model attends across both query and document tokens — full cross-attention
- No vector index can be built; the score must be computed fresh for every (query, doc) pair at inference
# Bi-Encoder vs Cross-Encoder
| Bi-Encoder | Cross-Encoder | |
|---|---|---|
| Input | query and doc encoded separately | query + doc concatenated |
| Speed | fast (pre-compute doc embeddings) | slow (re-encode every pair) |
| Accuracy | lower | higher |
| Scalable to large corpora | yes | no |
| Used for | retrieval | reranking |
# Papers
- Beyond the Reranker
- 2026-07-03 — once a strong cross-encoder reranker is present, only HyDE query expansion and per-source calibration (SSCC) give reliable gains on heterogeneous corpora