📑 RAG and Beyond

This paper provides a comprehensive survey of the current state of data-augmented Large Language Models (LLMs), focusing on Retrieval-Augmented Generation (RAG) and beyond. The authors classify different types of queries that utilize external data into four levels based on their complexity: explicit fact queries, implicit fact queries, interpretable rationale queries, and hidden rationale queries. They discuss the specific challenges associated with each level and provide a detailed overview of the most effective techniques for addressing them, such as RAG, prompt tuning, in-context learning, and fine-tuning. The paper ultimately aims to guide developers in systematically developing data-augmented LLM applications by offering solutions to the various challenges faced at each query level.

📎 Link to paper