AI Security
Phase 7 · Weeks 24–26LLM apps can be attacked with plain text. Learn the attacks, then the defences.
In an LLM app, any text the model reads can try to give it orders: a user message, a web page, an email, a PDF, a tool result. This area starts with the attacks (prompt injection, jailbreaks, data leakage) and the core defences (guardrails, validation, least privilege, sandboxing). The production half covers secrets, authorisation for agents, approvals, moderation, poisoning, RAG access control, MCP trust, audit logs and red teaming. Read 01–04 before Phase 3 gives your agent real tools.
0 of 20 used in code
01–10 · The base
Learn these first.
- 01Prompt InjectionUser input that overrides your instructions (‘ignore all previous instructions…’).
- 02Indirect Prompt InjectionHidden instructions inside web pages, emails or files that the model reads.
- 03JailbreaksPrompts written to get around a model’s safety rules.
- 04OWASP Top 10 for LLMsThe standard list of the biggest risks in LLM apps. Use it as your checklist.
- 05Input GuardrailsCheck and filter what goes into the model.
- 06Output ValidationCheck what comes out (schema, allowed values, safe content) before you use or show it.
- 07PII RedactionMask personal data (names, phone numbers, IDs) before you log it or send it to a model.
- 08Data LeakageThe model revealing your system prompt, other users’ data or secrets.
- 09Least-privilege ToolsEach tool and agent gets only the permissions it really needs.
- 10Tool SandboxingRun code or commands written by the model in an isolated environment.
11–20 · Production level
Come back after you have shipped something.
- 11Secrets ManagementKeys live in a vault or environment variables. Never in prompts, code or logs.
- 12API Key RotationReplace keys on a schedule, and immediately after any leak.
- 13Authorization for AgentsThe agent acts with the user’s permissions, checked on every action.
- 14Human Approval for ActionsA person confirms payments, deletes, emails and other steps you can’t undo.
- 15Content ModerationDetect and block harmful content going in and coming out.
- 16Data PoisoningBad data planted in training sets or in your RAG documents to change the model’s behaviour.
- 17Access Control in RAGUsers only retrieve documents they are allowed to see. Filter at search time.
- 18MCP Server TrustConnect only MCP servers you trust. A bad one can inject instructions or steal data.
- 19Audit LoggingA record of who asked what, and which tools ran with which arguments.
- 20Red TeamingAttack your own app on purpose to find the holes before someone else does.
Red-team your own agent with 20 injection and jailbreak prompts. Then add guardrails, output checks, tool permissions and an approval step, and test again.
Project guide →