arXiv · 2023 · Preprint
Gao et al. · → Paper · Demo: ? · Code: ?
A comprehensive survey of Retrieval-Augmented Generation (RAG) for large language models, introducing a three-paradigm taxonomy (Naive RAG, Advanced RAG, Modular RAG) and systematically reviewing retrieval, generation, and augmentation techniques across more than 100 RAG studies.
Citation Stub
This paper is not a speech generation paper but is cited by the corpus. See Scope and Coverage below for why it is relevant.
Scope and Coverage
This survey covers RAG methods from their origins alongside Transformer-based pre-training models through the ChatGPT era and into LLM fine-tuning integration, spanning the full development arc of RAG up to late 2023. The paper introduces a three-stage taxonomy as its primary organising scheme: Naive RAG (index-retrieve-generate pipelines), Advanced RAG (pre-retrieval and post-retrieval optimisation strategies), and Modular RAG (flexible, reconfigurable module compositions that support iterative, adaptive, and parallel retrieval flows). Within each paradigm, the survey covers the tripartite foundation of RAG: retrieval (sources, granularity, indexing optimisation, query optimisation, embedding, and adapters), generation (context curation and LLM fine-tuning), and augmentation (pre-training, fine-tuning, and inference-stage integration). The survey also reviews evaluation methodology, covering 26 downstream tasks, approximately 50 datasets, and current benchmarking tools. Speech generation systems that use language model backbones capable of external knowledge access, or that require factual grounding for knowledge-intensive spoken dialogue, can draw on this survey as a reference for how to structure and evaluate retrieval-augmented pipelines built on top of frozen or fine-tuned LLMs.
Wiki Connections
- spoken-language-model — RAG pipelines can extend spoken LMs by grounding generation in external knowledge, reducing hallucination in knowledge-intensive dialogue tasks.
- 2001.08361 (Scaling Laws for Neural Language Models) — cited to contextualise how large-scale pre-training shapes LLM knowledge capacity and motivates external retrieval as a complement to parametric knowledge.