GuidesAI development
Chunking documents for a RAG
The four rules
- 1Split at headingsOne section, one passage, as long as it stays short.
- 2Cap the sizeA section that is too long is split again at paragraphs.
- 3OverlapA few sentences shared between neighbouring passages.
- 4Keep the contextDocument and section titles added to each passage.
By document type
| Document | Method |
|---|---|
| Web pages, Markdown | Split by headings |
| Text PDF | Extract, then split by headings or paragraphs |
| Scanned PDF | Text recognition first |
| Tables | Kept whole, in Markdown |
| FAQ | One question and its answer per passage |
The metadata to keep
- The source and page, to cite.
- The language and date, to filter.
- Access rights, so a passage is never shown to someone who must not see it.
Testing your chunking
Take twenty real questions and look at the five passages returned for each. If the right answer is not there, change the chunking before touching the model.
Next: a RAG with pgvector and embeddings.
Frequently asked questions
- What passage size should I choose?
- A few hundred tokens is a common starting point. Too short and the passage loses its meaning; too long and it dilutes the search. Test on your real questions.
- Why overlap passages?
- So that an idea cut in two is found whole in at least one passage. An overlap of 10 to 20% is usually enough.
- How should tables be handled?
- Keep them whole, converted to Markdown, with the sentence that introduces them. Cutting a table in the middle makes it unreadable.