The idea
These solve different problems. RAG supplies knowledge at request time: it is easy to update (re-index a document), supports citations and per-user permissions, and works with any model. Fine-tuning changes the model’s behaviour: tone, format, domain vocabulary, or a narrow task done cheaply by a small model.
Ask: is the failure "the model doesn’t know X" (use RAG) or "the model doesn’t behave the way I need, even with good prompts and examples" (consider fine-tuning)? Fine-tuning facts in is brittle: they go stale, cannot be cited and may be hallucinated.
They combine well: a fine-tuned small model that is good at using retrieved context in your output format.
Choosing wrong wastes weeks. It is also a classic interview question.
Words to know
- Parametric knowledgeKnowledge stored in the model’s weights.
Try it without code
Classify five needs
- Write 5 requirements, e.g. "answers about this week’s prices", "always replies in our JSON schema".
- Mark each RAG, fine-tune, prompt, or both.
Use it in code
A decision helper encoding the main questions.
def recommend(changes_often, needs_citations, per_user_access, behaviour_problem, high_volume_narrow_task):
plan = []
if changes_often or needs_citations or per_user_access:
plan.append("RAG")
if behaviour_problem or high_volume_narrow_task:
plan.append("fine-tune (after prompts + examples fail)")
return plan or ["prompting + few-shot examples first"]
print(recommend(changes_often=True, needs_citations=True, per_user_access=False,
behaviour_problem=False, high_volume_narrow_task=False))Write a one-page decision record for a real feature explaining RAG vs fine-tuning vs both.
Common mistakes
- Fine-tuning on documents to "teach facts".
- Skipping prompt work before fine-tuning.