PAPERS-01-013 · Paper — Retrieval-Augmented

The paper

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, by Patrick Lewis and colleagues at Facebook AI Research (FAIR), published at NeurIPS 2020 (arXiv:2005.11401). It gave a name — RAG — to a now-ubiquitous pattern: pair a neural retriever with a sequence-to-sequence generator so the model can look facts up before it writes. The paper is the bridge between the closed-book language models that came before it and the source-citing assistants that came after.

Before this paper

By 2020 large pretrained models like B.E.R.T. and the early G.P.T. line had shown that a shocking amount of world knowledge could be baked into the weights themselves — a closed-book model answers from memory alone. But that parametric memory has hard limits: it is expensive to update, it cannot tell you where an answer came from, and it hallucinates confidently when the fact was never memorized or has since changed. The gap was clear — models knew a lot, but their knowledge was frozen at training time and impossible to audit.

The key idea

Split memory in two. Keep the model's learned parametric knowledge — the weights — but add an explicit non-parametric memory: a searchable index of real documents. When a question comes in, first retrieve the passages most likely to contain the answer, then generate the answer conditioned on those passages. The retriever and generator are trained together, so the model learns to write answers that are grounded in what it just fetched rather than in what it happened to memorize.

How it works

The retriever is Dense Passage Retrieval: every Wikipedia passage is encoded once into a vector and stored in an index; the question is encoded into the same space, and a fast nearest-neighbor search returns the top passages by inner product. Those passages are handed to a seq2seq generator (a B.A.R.T. model), which reads the question plus the retrieved text and produces the answer token by token. The paper studies two mixing rules — RAG-Sequence uses one retrieved document for the whole answer, while RAG-Token can attend to a different document for each token — and the whole system is trained end to end with only the retriever's query encoder and the generator updated.

See it — retrieve, then generate

flowchart LR
  Q["Question"] --> QE["Query<br/>encoder"]
  QE --> IDX["Dense index<br/>(Wikipedia)"]
  IDX --> TOP["Top-k<br/>passages"]
  TOP --> GEN["Seq2seq<br/>generator"]
  Q --> GEN
  GEN --> A["Grounded<br/>answer"]

What they showed

On several open-domain question-answering benchmarks, RAG set a new state of the art, outperforming the closed-book seq2seq models it was built from. Beyond the leaderboard numbers, the paper showed that the retrieved grounding made generations more specific and more factual — answers that pointed at real evidence rather than fabricating it. And because the knowledge lives in a swappable index, they demonstrated you can update what the model knows by replacing the document store, with no retraining of the weights at all.

What it changed

RAG turned "grounding a generator in a corpus" from a research curiosity into the default architecture for knowledge-intensive applications — the pattern behind essentially every modern "chat with your documents," enterprise search, and citation-producing assistant. It reframed hallucination and staleness as retrieval problems you can engineer against, not fixed properties of a frozen model. Grounding solves knowledge; the last piece is adapting a giant model cheaply. Next: Low-Rank Adaptation (L.o.R.A.).

The one-line takeaway

Don't memorize everything — retrieve, then generate, and the answer is grounded and updatable.

Read it / meet it

Read the paper at arxiv.org/abs/2005.11401. The idea lives today in nearly every production LLM assistant that cites sources or answers over a private knowledge base — vector databases, embedding indexes, and the retrieval pipelines that feed an LLM its context are all direct descendants of this design.

Named concepts: retrieval-augmented generation, dense passage retrieval, open-domain question answering, parametric vs non-parametric memory