Area 05 · AI System Design · Concept 18 of 20

RAG vs Fine-tuning

RAG for fresh or private facts. Fine-tuning for style, format or one narrow skill.

11–20 · Production levelPhase 6 · Weeks 21–23≈ 45 min

The idea

These solve different problems. RAG supplies knowledge at request time: it is easy to update (re-index a document), supports citations and per-user permissions, and works with any model. Fine-tuning changes the model’s behaviour: tone, format, domain vocabulary, or a narrow task done cheaply by a small model.

Ask: is the failure "the model doesn’t know X" (use RAG) or "the model doesn’t behave the way I need, even with good prompts and examples" (consider fine-tuning)? Fine-tuning facts in is brittle: they go stale, cannot be cited and may be hallucinated.

They combine well: a fine-tuned small model that is good at using retrieved context in your output format.

Why it matters

Choosing wrong wastes weeks. It is also a classic interview question.

Words to know

  • Parametric knowledgeKnowledge stored in the model’s weights.

Try it without code

Classify five needs

  1. Write 5 requirements, e.g. "answers about this week’s prices", "always replies in our JSON schema".
  2. Mark each RAG, fine-tune, prompt, or both.

Use it in code

A decision helper encoding the main questions.

python
def recommend(changes_often, needs_citations, per_user_access, behaviour_problem, high_volume_narrow_task):
    plan = []
    if changes_often or needs_citations or per_user_access:
        plan.append("RAG")
    if behaviour_problem or high_volume_narrow_task:
        plan.append("fine-tune (after prompts + examples fail)")
    return plan or ["prompting + few-shot examples first"]

print(recommend(changes_often=True, needs_citations=True, per_user_access=False,
                behaviour_problem=False, high_volume_narrow_task=False))
PRACTICE · TICK IT AFTER THIS

Write a one-page decision record for a real feature explaining RAG vs fine-tuning vs both.

Common mistakes

  • Fine-tuning on documents to "teach facts".
  • Skipping prompt work before fine-tuning.

Go deeper