Dense Retrieval (Bi-Encoder)

What is the difference between sparse and dense retrieval?:

Dense retrieval, on the other hand, uses neural networks to map text into lower-dimensional, continuous vectors (embeddings) that capture semantic meaning. These vectors are “dense” because every dimension contains a non-zero value, allowing similarities to be measured even when the exact keywords don’t match

Encoder or embedding model is used to transforms text chunks into a vector representation upon which the search is performed. This note aggregates more practical rather than theoretical knowledge related to encoders. For theoretical information see BERT and Transformers.

• Embedding Serving
• Bi-Encoder Fine-tuning
• Encoder Libraries
• Embedding Leaderboards

• ON THE THEORETICAL LIMITATIONS OF EMBEDDING-BASED RETRIEVAL
  • 2026-03-12
  • 50 citations
• The Universal Weight Subspace Hypothesis
  • 8 citations
  • 2025-12-04

• Recent Trends with Text Embeddings Decoder-Only LLMs
  • 2024-12-02 — reviews E5-Mistral, NV-Embed, GritLM, LLM2Vec and the decoder-only shift on the MTEB leaderboard