QA-RAG

Good summary of this method The Geometry of Queries: Query-Based Innovations in Retrieval-Augmented Generation:

QA-RAG. In their methodology, they fine tune a foundation LLM offline using a golden set of Q&A. Upon receiving a new query online, they generate a pseudoanswer using the fine tuned LLM. They then retrieve contents using the query embeddings only, and other contents using the pseudo-answer’s embeddings only. A Reranker is then deployed to select only the most relevant contents among this dual set. In our implementation

Paper:

However, Multiquery retrieval is limited as it is still confined to the narrow scope of the user’s question, hindering its ability to capture a wide range of information. Also, in domain-specific and highly specialized areas like pharmaceutical regulatory compliance, using a general LLM like what has been done in HyDE often produces very incomplete hypothetical answers, necessitating the employment of a more specialized approach. Recognizing this, we utilized a finetuned LLM that has been trained on domain-specific data, which enabled it to generate responses with a level of detail and accuracy akin to that of the expert in the pharmaceutical field. Half of the documents were retrieved using the answers provided by this fine-tuned LLM. To enhance the diversity of search, the other half is sourced using the user’s question. By utilizing both the user’s query and the tailored responses generated by the fine-tuned LLM, this dual-track approach achieved a more thorough and nuanced retrieval of information.