LEAD-01-023-P · Primer — Knowledge and Retrieval (RAG)

The idea in one sentence

Retrieval-augmented generation grounds an agent in your real data by fetching the relevant documents at question-time and forcing the agent to answer from them, instead of from whatever it half-remembers.

Why it exists

A base model knows the public internet up to a cutoff date and nothing about your pricing, your policies, or last Tuesday's contract — so on your questions it confidently guesses. RAG turns the agent from a know-it-all consultant into a diligent analyst who looks it up before answering, with the source cited. For any regulated or high-stakes use, 'where did that answer come from?' must have a document behind it, not a vibe.

How it works

1. Your documents are chunked and indexed (often as vector embeddings so the system can search by meaning, not just keywords). 2) At question-time the agent retrieves the handful of most relevant chunks. 3) Those chunks are placed in the agent's context as grounding. 4) The agent answers from that evidence and cites it. 5) Evaluation checks not just 'is the answer fluent' but 'is it faithful to the retrieved source' — because the classic failure is a confident answer citing a policy that never existed. Retrieval quality caps everything: if the right document is not retrieved, no amount of model cleverness recovers it, so the boring work of chunking, freshness, and access-scoped indexes is where the value is. Example: a customer-support agent that answers refund questions must pull the current refund policy and quote it — not recite a plausible-sounding policy from training data.

See it

flowchart LR
  A["Question"] --> B["Retrieve<br/>Relevant Docs"]
  B --> C["Your Indexed<br/>Knowledge"]
  C --> B
  B --> D["Answer from<br/>Evidence"]
  D --> E["Cite Source"]
  D --> F["Faithfulness<br/>Check"]
  F -->|unsupported| B

In practice

Wealth management: an advisor-support agent answers 'can this client hold this fund in their IRA?' by retrieving the current product prospectus and the account's eligibility rules, quoting the exact clause, and flagging when the documents disagree — rather than improvising a rule that sounds right.

The one thing to remember

An ungrounded agent is a confident liar about your business; retrieval is what makes it look things up before it speaks.