These are not competing answers to the same question, which is why the comparison confuses so many projects. Retrieval solves a knowledge problem: the model does not know your refund policy. Fine-tuning solves a behaviour problem: the model knows the answer but will not produce it in the shape your system needs.
Start with retrieval, because it fails visibly
When RAG is wrong you can usually see why — the cited passage is the wrong one, or nothing was cited at all. When a fine-tuned model is wrong it is wrong fluently, in exactly the format you asked for, and the mistake surfaces only when somebody checks the underlying fact. For business knowledge that changes, retrieval is the safer default and the cheaper thing to correct.
Neither fixes bad source material
A retrieval system over three contradictory versions of the same policy will answer confidently from whichever one it found. A fine-tune trained on inconsistent examples learns the inconsistency. The unglamorous prerequisite for both is a reviewed, current, permission-aware set of source documents — which is usually where the real work of a knowledge project sits.
The two also combine: retrieval supplies the facts, a light fine-tune enforces the output contract. That is a second-phase optimisation, not a starting point.
01Real scenario
A support team needs answers from policies that change monthly and also wants replies in a consistent format. Retrieval addresses the changing facts; fine-tuning may later standardise behaviour, but the two jobs must be evaluated separately.
02Inputs and evidence
Prepare approved source documents, access-control groups, a reviewed question set and examples of the desired output format. Record which failures come from missing knowledge and which come from inconsistent behaviour.
03Actions to test
Build retrieval first, require citations and evaluate whether the correct passage is found. Consider fine-tuning only after the factual path works and a large, clean example set demonstrates a stable behaviour problem.
04Security and controls
Propagate source permissions before retrieval, log cited passages and refuse when evidence is missing. Training data needs provenance, sensitive-data review, versioning and a rollback path for every model release.
05Pilot KPI
Score retrieval relevance, citation correctness, grounded-answer quality, safe refusals, latency and cost per question. For fine-tuning, separately score format compliance and behaviour consistency on an untouched evaluation set.
06Errors and human hand-off
Stop when sources conflict, permissions leak, citations do not support the answer or the fine-tune improves style while factual accuracy falls. Hand unresolved questions to the knowledge owner with the retrieved evidence attached.