GuidesAI reliability
Prompt injection: understanding the attack and defending against it
Two forms
Direct
Indirect
Defenses
| Defense | Effect |
|---|---|
| Least privilege | A support agent cannot issue refunds |
| Human confirmation | Every irreversible action waits for a click |
| Outside content tagged | The prompt states this text is data, not an instruction |
| Validated output | Schema, allowlist of URLs and recipients |
| No secrets in the prompt | Nothing to leak |
| Logging | Trace what triggered an action |
The golden rule
- 1ReadThe agent may read untrusted content…
- 2Act…or act with significant rights…
- 3Not both without a safeguardOtherwise a human confirmation steps in.
Background: AI agents on real data and tool use. Testing: test an AI chatbot.
Frequently asked questions
- Is a good system prompt enough?
- No. It reduces the risk without removing it. The real protection lies in the architecture: what the agent is allowed to do, and what requires confirmation.
- Are recent models resistant?
- More than before, thanks to their training. Injection still tops the OWASP Top 10 for LLM applications.
- Which agents are most exposed?
- Those that read outside content (web, emails, received documents) and can act (send, pay, delete).