GuidesUnderstanding AI

RAG or fine‑tuning: which one for your AI

By the Banzai Labs teamUpdated on 3 min read

At answer timeThe model is retrainedRAG: hand it your documentsFine-tuning: change it
RAG hands your documents to the model at each question. Fine-tuning changes the model itself.

The difference in one line

Two ways to bring in your knowledge
RAG
  • Search your documents
  • Attach the passages to the question
  • The model does not change

Good for facts that move.

Fine-tuning
  • Gather examples
  • Retrain the model
  • The model changes

Good for a style, a format, a repeated task.

The decision table

Criterion RAG Fine-tuning
Changing data Updated by re-indexing New training run
Cited source Yes No
Imposed tone, format Through the prompt Learned
Upfront cost Low Example set to build
Running cost More input tokens Shorter prompts

How to decide

In this order
  1. 1A good promptOften enough for tone and format. See the system prompt.
  2. 2RAGAs soon as the answer depends on your documents.
  3. 3Fine-tuningIf a very precise format or a huge volume justifies it.

Further reading: RAG, AI on your documents and embeddings.

Frequently asked questions

Does fine-tuning teach the model my documents?
Poorly. It mostly sets a way of answering. For precise, up-to-date facts, RAG is more reliable and cites its source.
Can the two be combined?
Yes. A model tuned for tone and format, fed by RAG for facts. It is rarely needed at the start.
Where to start?
With a good prompt and RAG. Fine-tuning comes later, if a style or format need remains poorly covered.

Read next

All guides