Area 06 · AI Security · Concept 14 of 20

Human Approval for Actions

A person confirms payments, deletes, emails and other steps you can’t undo.

11–20 · Production levelPhase 7 · Weeks 24–26≈ 45 min

The idea

Even with least privilege and authorisation, some allowed actions are too consequential to leave to the model alone: money movement, deletion, external communication, production changes. Approval puts a human decision between the model’s intent and the effect.

Make approval meaningful: show exactly what will happen (recipient, amount, full text), highlight anything unusual (new recipient, large amount), and make rejection easy. Rubber-stamp fatigue is real, so only gate actions that matter.

Enforce it in code at the tool layer and log who approved what.

Why it matters

It is the last line of defence when injection or a model mistake gets through every other layer.

Words to know

  • Approval fatiguePeople approving without reading because prompts are too frequent.

Try it without code

Design the approval card

  1. Sketch the approval screen for "send email".
  2. Include recipient, subject, full body, and a warning if the recipient is external.

Use it in code

Risk-scored approval: only unusual actions need a human.

python
def needs_approval(call, user):
    if call.name == "send_email":
        to = call.input["to"]
        return not to.endswith("@ourcompany.ie") or to not in user.known_contacts
    if call.name == "refund":
        return call.input["amount_eur"] > 50
    if call.name in {"delete_file", "deploy"}:
        return True
    return False

def describe(call):
    return "\n".join(f"{k}: {v}" for k, v in call.input.items())   # show everything, verbatim
PRACTICE · TICK IT AFTER THIS

Add risk-scored approval to your agent and log approvals and rejections with user and timestamp.

Common mistakes

  • Approval prompts that summarise instead of showing exact arguments.
  • Approving everything, so people stop reading.

Go deeper