GuidesRAG
RAG: getting an AI to answer from your documents
The problem RAG solves
A language model does not know your contracts, your documentation or your prices. If you ask anyway, it answers with what seems plausible. It is wrong, and stated with confidence.
RAG changes the order of operations: search first, write second.
How it works
- 1SplitYour documents into passages.
- 2IndexThose passages in a search engine, often a semantic one.
- 3For each questionRetrieve the closest passages.
- 4Give them to the modelWith an instruction: answer only from them, and cite.
- 5The answerArrives with its sources.
The model no longer serves as memory. It serves as a writer, on documents just placed in front of it.
What makes the quality: the documents
Nine problems out of ten come from the content, not the model.
- Up-to-date documentsTwo contradictory versions give two contradictory answers.
- Splitting that respects meaningA passage cut in the middle of a rule means nothing.
- MetadataDate, author, product, language. It allows filtering.
- Tables and PDFs handled with careA badly extracted table becomes noise.
What makes the quality: the search
- Semantic and keyword search togetherThe first understands meaning, the second finds an exact reference.
- Re-rankingOf results before giving them to the model.
- A thresholdBelow a certain relevance, no answer is given.
Saying “I don’t know”
It is the most important rule. When no passage answers, the system says so and offers another route. A missing answer does less damage than an invented one.
On Votons, the assistant answers questions about the manifestos by relying on the exact text. On a topic that sensitive, the displayed source is part of the answer.
Access rights
An enterprise RAG must respect who is allowed to read what. The search filters by user: a salesperson must not get an extract from an HR document. This filtering is designed at indexing time.
Measuring quality
- A set of real questions with the expected answer and the source document.
- Two measures: was the right passage found, and is the answer faithful to the passage.
- The set is replayed at every change of model, splitting or instruction.
When RAG is not enough
- Questions that require a calculation on data: you need a tool, not passages.
- Questions that require reading a whole corpus: a prepared summary does better.
- Actions: to act, you need an agent with tools. See AI agents and MCP server.
The most common use of RAG is described in AI chatbot for your website.
Frequently asked questions
- What does RAG stand for?
- Retrieval-Augmented Generation. The system first searches a document base for relevant passages, then asks the model to answer from those passages.
- Do you need to train a model on your documents?
- No. With RAG, the model is not modified. Your documents are supplied at question time. Updating them is enough for the answers to change.
- Does RAG remove AI errors?
- It reduces them sharply when the search finds the right passage. You still have to handle cases where nothing matches: the system must then say it does not know rather than fill the gap.
