Retrieval-Augmented Generation (RAG)

How RAG works

A language model's built-in knowledge is frozen at training time and can't include your private documents or today's facts. RAG fixes this by adding a retrieval step before generation: the system takes the user's question, searches a knowledge base — often a vector database of embedded documents — for the most relevant passages, and inserts those passages into the model's prompt as context. The model then answers using that retrieved material rather than guessing from memory.

The payoff is grounding. Because the answer is built from specific, retrieved sources, the model is less likely to hallucinate and can cite where each claim came from. This is why research agents like Elicit and GPT Researcher feel trustworthy — they retrieve real papers or web pages and synthesize from them, rather than producing unsourced prose.

Why it matters for agents

RAG is what lets an agent work over knowledge it was never trained on — a company's internal wiki, a product's documentation, a live help center, or a corpus of academic papers. Customer-support agents use it to answer from your help articles and policies; research agents use it to read sources on demand. Without retrieval, an agent is limited to general knowledge and prone to confident errors on anything specific.

RAG isn't a complete safeguard, though. The answer is only as good as what's retrieved — poor search returns poor context — and retrieved content is also a vector for prompt injection if it isn't treated as untrusted. Good RAG systems invest as much in the quality and security of retrieval as in the model itself.

What is retrieval-augmented generation (RAG)?

RAG is a technique where an AI model retrieves relevant documents from an external knowledge source and uses them as context to ground its answer, instead of relying only on training data. It lets agents answer from current, private, or specialized information and cite sources.

Why does RAG reduce hallucination?

Because the model builds its answer from specific passages retrieved from a real knowledge base rather than guessing from memory, it's less likely to invent facts and can point to where each claim came from. The grounding is only as good as the retrieval, though.

How do AI agents use RAG?

RAG lets an agent work over knowledge it wasn't trained on — a help center, internal docs, or academic papers. Support agents answer from your policies and research agents read sources on demand, which is what makes their answers specific and citable.

What is the difference between RAG and fine-tuning?

RAG adds knowledge at query time by retrieving documents into the prompt; fine-tuning bakes it into the model's weights by training. RAG is cheaper, updates instantly when your documents change, and can cite sources — which is why most knowledge-based agents use it. Fine-tuning suits teaching a model a style, format, or task, not facts that change.

What is an example of RAG?

A support agent like Intercom's Fin: you ask a question, it searches your help centre and past conversations, pulls the relevant passages, and answers from those rather than from general training data. Research agents work the same way — Elicit retrieves actual papers and extracts findings from them, so every claim traces to a source.

Is RAG still needed now that models have long context windows?

Yes, for most real corpora. You can't fit a whole help centre, document store, or 125M-paper corpus in a context window, and stuffing everything in is slow and expensive even when it fits. Retrieval selects the relevant few passages, which is cheaper, faster, and keeps the citation trail — long context complements RAG rather than replacing it.

What is agentic RAG?

RAG where an agent controls the retrieval loop rather than doing one fixed search: it decides what to look for, evaluates what came back, reformulates the query, and searches again until it has enough to answer. GPT Researcher works this way — planning queries, reading results, and refining — which is why it produces a report rather than a single answer.