Multi-Vector Retrieval

• The multi-vector search can be used for special use cases when an entity within the database is represented by multiple vectors. An example of this can be a house listing that is represented by multiple pictures. You can then represent this house listing with multiple vectors (embeddings of pictures) and increase the chance of finding it when performing the search.
• This operation might seem quite exotic, but it is already being used in RAG applications. We can use this search operator to take advantage of late embedding with Contextualized Late Interaction over BERT (ColBERT). This embedding approach can be used as a reranker (Vespa ColBert Reranking) or as a default embedding. Using Colbert can drastically improve the quality of search at the cost of search time.\
• 💡Multi-vector can be employed with ColBERT to perform reranking or with the ColBERT as default embedding to improve the search quality at the cost of search speed.

• MUVERA (Multi-Vector Retrieval with Approximation)
• PLAID
  • PLAID: An Efficient Engine for Late Interaction Retrieval
• GEM
  • 0 citations
  • 2026-03-26
  • Graph-based index natively designed for multi-vector (Chamfer/MaxSim) retrieval; decouples graph construction (EMD) from search (Chamfer), injects semantic shortcuts, and achieves up to 16x speedup over PLAID/DESSERT/MUVERA

Section titled “Storage Considerations”

Multi-vector embeddings are significantly larger than dense embeddings because they store one vector per token rather than a single pooled vector:

The multi-vector storage calculation: 1M docs x 100 tokens x 128 dims x 4 bytes = ~51 GB. Actual token counts vary by document length (max 512 for most models, 8192 for long-context models).

Section titled “Multi-Vector Models”