Capstone: threat-model an AI application

P59.ai-threat-modeling.01 · Audience: guest, it-ml, language-pro · Prerequisites: Data leakage & model-side risks, Tool & permission boundaries for agents, Sandboxing & isolation

Real LLM grading for this pageLLM grading (this page):

Every defence in this pillar was built one rung at a time — quarantine, redaction, permission gates, sandboxes — but real systems are attacked as a whole, so this capstone asks you to defend one as a whole: a realistic multi-tenant customer-support agent that retrieves from a shared knowledge base and the user's own tickets, and can call tools including send_email and issue_refund. Your deliverable is its threat model: you map assets and trust boundaries, enumerate abuse cases with STRIDE — a checklist of six threat categories, from spoofing to elevation of privilege, adapted here to LLM/agent systems — and attach to each threat a mitigation you built in P59. You end by stating the residual risk honestly, because no injection defence is complete — which is exactly why the irreversible actions sit behind a human. No code runs and nothing asks you to attack a system: this is a design exercise, and the deliverable is the defence plan.

Ask the mentor about this module

Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.

Images, PDF or text. Kept on this device only.
Keeping your files on this device

Off by default. The mentor always gets your file; this only decides whether your own copy stays here. Copies live in this browser only - they do not follow you to another device, and clearing site data removes them.

Ctrl/Cmd + Enter to send
Capstone — threat-model an agent application (free-form)

Loading exercise…

My notes on this module

Loading your notes...

Capstone: threat-model an AI application — TransformerLab