An AI answering from memory can be wrong. One that checks the right page first is much more reliable. That's RAG.
In a closed-book exam, you answer from memory. In an open-book exam, you look up the right page first, then write your answer. You’re far less likely to make things up.
RAG gives an AI an open book. Before answering, the AI searches a set of documents for the most relevant bits, then writes its answer using them. The name stands for Retrieval-Augmented Generation: fetch, then write.
How it works
- you ask a question
- a search finds the few passages most likely to help
- those passages go into the AI's context window along with your question
- the AI answers, ideally pointing to where it found each fact
Why companies love it
It lets an AI answer questions about your own policies, products or contracts, without retraining it. Update the documents and the answers update too.
What it isn’t
It isn’t a cure for hallucination. It makes it much rarer, but the AI can still misread a passage or fill in gaps.
Where the comparison breaks down
A student chooses which page to open. In RAG, a separate search step picks the pages. If it picks the wrong ones, the AI answers from the wrong pages.
How the words fit together
RAG fetches documents into the context window, often using embeddings to find them. It makes LLMs less likely to hallucinate.
RAG answers are only as good as the documents behind them. Out-of-date or messy files mean out-of-date or messy answers.

When you'll hear it
“We use RAG over our docs.” The AI searches our own documents first, then answers from what it found.
“It cites its sources.” The answer comes with links to the pages it used, so you can check.
“The retrieval missed it.” The search step didn't find the right document, so the answer was poor.
“If we put an AI in front of our documents, who keeps those documents up to date?”


