The idea
Metrics tell you how often it fails; error analysis tells you why. Collect 50–100 real outputs (from logs or a golden run), read each one, and write a short note on what went wrong. Then group the notes into categories: wrong retrieval, ignored instruction, bad format, hallucinated fact, refused wrongly.
Count each category. Usually two or three categories cause most failures. Fix the biggest one, rerun, and repeat. This beats guessing at prompt tweaks.
Do it by hand first. Once categories are stable, you can write automated checks or judges for each.
It is the highest-return activity in AI engineering and the one most teams skip.
Words to know
- Failure modeA recurring type of mistake.
Try it without code
Open coding
- Export 30 outputs into a spreadsheet.
- Add a column "what went wrong" and fill it in plain words.
- Add a second column grouping them into 4–6 categories, then count.
Use it in code
Turn your hand-written notes into a failure table.
import csv
from collections import Counter
# review.csv columns: id, output, ok (y/n), failure (category name or blank), note
rows = list(csv.DictReader(open("review.csv")))
fails = [r for r in rows if r["ok"] == "n"]
print(f"{len(fails)}/{len(rows)} failed")
for cat, n in Counter(r["failure"] for r in fails).most_common():
print(f"{n:3d} {cat}")Review 50 outputs from your RAG app, categorise failures, fix the top category and report the before/after count.
Common mistakes
- Jumping to automated metrics before reading outputs.
- Categories so broad ("bad answer") that they do not suggest a fix.