TEACH THE FACTS OR THE BEHAVIOUR

RAG vs fine-tuning for business knowledge

Retrieval gives a model access to your current documents; fine-tuning changes how it behaves. A comparison of freshness, traceability, permissions and cost.

Discuss your case

RAG (retrieval)

The model answers from documents fetched at question time, with citations back to the source.

Fine-tuning

The model's weights are adjusted on examples so it consistently produces a particular format or behaviour.

Criterion	RAG (retrieval)	Fine-tuning
What it changes	What the model can see	How the model responds
Knowledge freshness	As current as the last document sync	Frozen at training time; new facts need retraining
Traceability	Answers can cite the source passage	No source; the answer comes from the weights
Permissions	Retrieval can respect per-user document access	Training data is baked in for everyone
Cost to update	Re-index the changed documents	Re-run training and re-evaluate
Setup effort	Source inventory, chunking, retrieval, evaluation	A curated, labelled example set
Typical failure	Right answer missing because retrieval missed it	Confident output in the right shape but the wrong facts
Per-request cost	Higher — retrieved context is sent every time	Lower — shorter prompts once behaviour is learned

Choose RAG (retrieval) when

The facts live in documents that change
Answers must be attributable to a source
Different users may see different documents
You need it working before you have a labelled dataset

Choose Fine-tuning when

The task is a stable, repeated transformation or classification
Output format or tone must be identical every time
Prompt length or latency is a real constraint at volume
You have hundreds of clean, reviewed examples of the correct output

These are not competing answers to the same question, which is why the comparison confuses so many projects. Retrieval solves a knowledge problem: the model does not know your refund policy. Fine-tuning solves a behaviour problem: the model knows the answer but will not produce it in the shape your system needs.

Start with retrieval, because it fails visibly

When RAG is wrong you can usually see why — the cited passage is the wrong one, or nothing was cited at all. When a fine-tuned model is wrong it is wrong fluently, in exactly the format you asked for, and the mistake surfaces only when somebody checks the underlying fact. For business knowledge that changes, retrieval is the safer default and the cheaper thing to correct.

Neither fixes bad source material

A retrieval system over three contradictory versions of the same policy will answer confidently from whichever one it found. A fine-tune trained on inconsistent examples learns the inconsistency. The unglamorous prerequisite for both is a reviewed, current, permission-aware set of source documents — which is usually where the real work of a knowledge project sits.

The two also combine: retrieval supplies the facts, a light fine-tune enforces the output contract. That is a second-phase optimisation, not a starting point.