Chapter 11: Safety: Prompt Injection & Defense
How attackers hijack agents with malicious instructions hidden in data, the taxonomy of direct vs indirect injection, the data-tagging discipline that gives the model a notion of trust, output-validation firewalls, and red-teaming agents to find vulnerabilities before attackers do.
- Topics
- 5
- Videos
- 3
Chapter Overview
An LLM cannot tell the difference between an instruction from the developer (in the system prompt) and an instruction from the world (in a tool's output). Whatever text is in the model's context, the model treats as input to think about. Prompt injection weaponizes this: an attacker embeds 'IGNORE PRIOR INSTRUCTIONS AND...' in a webpage, an email, a document, or any other source the agent might read.
For agents — which read tool outputs constantly, often from untrusted sources — prompt injection is the #1 security concern. Simon Willison has called it 'the most important security problem in AI today,' and OWASP listed LLM01:Prompt Injection as the top LLM application risk.
This chapter covers:
- Prompt injection taxonomy — the categories of attack
- Direct vs indirect injection — who provides the malicious text
- Data tagging & trust boundaries — give the model a sense of where text came from
- Output validation & firewalls — catch bad agent actions before they leave the system
- Red-teaming agents — finding vulnerabilities before attackers do
Chapter Roadmap
Click any topic to jump in
Injection Taxonomy
What can be injected and how — the full attack surface for agent systems.
User-as-attacker vs third-party-as-attacker
Direct vs Indirect
Direct (user attacks) is mostly solved; indirect (third-party content) is the open problem.
Data Tagging
Mark untrusted text and instruct the model to treat it as data — necessary but not sufficient.
Output Validation
Validate every action before executing. Argument allowlists, volume limits, human-in-the-loop.
Red-Teaming
Probe systematically. Automated attackers + scored evaluators give you a security posture you can track.
Prompt Injection Taxonomy
Direct vs Indirect Injection
Data Tagging and Trust Boundaries
Sign up to unlock this chapter
This chapter is part of PixelBank Premium. Create a free account, then upgrade to read the full lesson — concepts, walkthroughs, and exercises.