Area 01 · LLM · Concept 16 of 20

Fine-tuning with LoRA

Train small adapter matrices instead of all the weights. A cheap way to specialise a model.

11–20 · Production levelPhase 1 · Weeks 3–6≈ 3 hours

The idea

Fine-tuning continues training a model on your own examples so it learns a style, format or narrow skill. Full fine-tuning updates billions of weights and needs a lot of GPU memory. LoRA (Low-Rank Adaptation) freezes the original weights and trains tiny extra matrices added to some layers, often well under 1% of the parameter count.

The result is a small adapter file, megabytes rather than gigabytes, that you load on top of the base model. You can keep several adapters for different tasks and swap them on the same base.

Fine-tuning teaches behaviour, not fresh facts. Use it when prompting and examples cannot get the format or tone reliable, or to make a small model do one job as well as a big model. For knowledge that changes, use RAG.

Why it matters

It is how teams cut cost (small specialised models) and get very consistent outputs where prompting plateaus.

Picture it

Words to know

  • AdapterThe small set of trained LoRA weights.
  • Rank (r)Size of the LoRA matrices; higher means more capacity.
  • PEFTParameter-efficient fine-tuning; LoRA is one method.

Try it without code

Decide if you need it

  1. List 3 tasks where your prompt still fails after adding examples.
  2. For each, write whether the failure is missing knowledge (use RAG) or missing behaviour (consider fine-tuning).

Use it in code

Attach a LoRA adapter with PEFT and see how few parameters train.

python
# pip install transformers peft
from transformers import AutoModelForCausalLM
from peft import LoraConfig, get_peft_model

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
config = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05,
                    target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM")
model = get_peft_model(base, config)
model.print_trainable_parameters()
# Next: train with transformers Trainer or TRL's SFTTrainer on your JSONL examples,
# then model.save_pretrained("my-adapter")
PRACTICE · TICK IT AFTER THIS

Fine-tune a small model with LoRA on 200 examples of one format (e.g. commit messages) and compare it with the base model on 20 held-out inputs.

Common mistakes

  • Fine-tuning to add facts that will change next month.
  • Training on fewer than ~100 clean examples and expecting magic.
  • No held-out test set.

Go deeper