AI agent security: how to defend against prompt injection

AI agent security starts from one simple assumption: the agent can be fooled, so you limit what it is able to do when that happens. No single safeguard stops prompt injection, but layered defenses (least privilege, isolation, guardrails, human approval and regular testing) sharply reduce the damage an attack can cause.

Why an agent is more exposed than a chatbot

A chatbot answers with text. If someone pushes it into saying something wrong, the harm is usually limited to a bad or inappropriate reply. An agent acts. It reads your email, browses web pages, opens files and uses tools through tool calling, meaning it triggers real functions: sending a message, writing to a database, placing an order.

Three ingredients make things risky when they come together:

1. Access to private data: an inbox, internal documents, a customer database.
2. Exposure to untrusted content: web pages, attachments, support tickets, online reviews, anything a stranger can write.
3. The ability to act or communicate externally: sending a message, calling a service, publishing.

An agent that combines all three can, if manipulated, leak information or trigger an action nobody wanted. If you are new to building agents, our complete guide to building an AI agent covers the foundations first. This article focuses on protection.

Prompt injection in plain English

Prompt injection means slipping instructions into text the AI will read, so that it follows them instead of the intended ones. It works because a language model receives everything as text. The developer’s instructions, the user’s request and the content the agent reads all arrive in the same stream, with no watertight line between “commands” and “data”.

There are two families:

• Direct injection comes from the person talking to the agent, who tries to make it ignore its rules.
• Indirect injection is hidden in external content the agent processes: a web page, a PDF, an email. It is the more worrying kind, because the legitimate user often cannot see it and triggers it without knowing.

We deliberately do not describe how these attacks are built. What matters for you is designing the agent so that a malicious instruction, even if followed, cannot do much harm.

The layers of defense to put in place

None of these measures is enough on its own. Their strength comes from stacking them: if one layer fails, the next one limits the consequences.

1. Grant the least privilege possible

This is the principle of least privilege. Every tool you hand the agent should have the narrowest scope possible: read-only when reading is enough, one folder rather than the whole drive, a test mailbox rather than your main inbox. Create dedicated credentials for the agent, revocable and time-limited, and never reuse your own. API keys belong on the server side, never in the text the model reads.

OWASP also advises handling sensitive functions in code rather than in the prompt. For example, instead of telling the agent “never refund more than $50”, make the refund tool itself reject any larger amount. If you expose your tools through MCP, every tool is a door: our tutorial on how to build an MCP server shows how to keep them narrowly scoped.

2. Run the agent in a sandbox

A sandbox keeps the agent inside a closed environment. Two barriers matter most: file system isolation (the agent only sees a working folder) and network isolation (it can only reach approved addresses). In an October 2025 article about isolating its Claude Code tool, Anthropic stressed that both are needed: an agent that can read sensitive data and contact any server can send that data out.

For agents that drive a browser or a computer, a dedicated virtual machine that is reset after each task is good practice.

3. Separate and label external content

Anything coming from outside should be marked as such and treated as data, not as instructions. A careful architecture splits the roles: one component reads untrusted content but has no access to sensitive tools; another one acts, but only receives a structured summary from the first. Clear instructions in the system prompt help the agent stay in its lane, but they guarantee nothing against well-crafted malicious text.

4. Filter inputs and outputs

Guardrails are checks placed around the model. On the way in, they flag out-of-scope requests or sensitive data. On the way out, they check the answer or action before it goes anywhere. Enforcing a strict output format makes this far easier: if the agent must return an amount and an order reference, your code can check with certainty that the amount stays within a range and that the reference exists. Well-known projects include NeMo Guardrails, an open-source toolkit from NVIDIA, and Llama Guard, a risk classification model released by Meta.

5. Keep a human in charge of irreversible actions

Human in the loop means a person approves certain actions before they run: payments, outbound messages, deletions, permission changes. Show that person the exact action and its parameters, not a summary written by the agent. Watch out for approval fatigue: when everything needs a click, people end up approving without reading. Keep confirmations for actions that matter and let the rest run in the sandbox.

6. Log everything and keep a kill switch

Record every tool call, its parameters and where the content the agent read came from. These logs help you spot unusual behavior, such as an abnormal number of outgoing messages or an unknown destination, and understand an incident afterwards. Plan an emergency stop: shutting the agent down and revoking its access should take seconds, not a meeting.

7. Test like an attacker, regularly

Red teaming means having a team or a tool try to hijack the agent in a test environment, so you find the weaknesses before anyone else does. Add security scenarios to your test suites, next to the quality tests described in our guide to evaluating an AI agent. Repeat after every change of model, tool or instructions: an agent that was safe yesterday may not be safe tomorrow.

The seven layers at a glance

Layer	What it limits	Concrete example
Least privilege	How much damage is possible	Read-only access to a single folder
Sandbox	Leaks and side effects	Network limited to an allowlist
Content separation	The influence of untrusted text	The page reader has no sending tool
Guardrails	Out-of-scope requests and answers	Automatic check on the returned amount
Human in the loop	Irreversible actions	Confirmation before any payment
Logging	Incidents nobody notices	Alert on an unknown destination address
Adversarial testing	Undiscovered weaknesses	Test campaign after each update

What vendors and frameworks provide

Companies building agentic products increasingly explain their protections. Perplexity, for instance, describes a multi-layer defense against prompt injection for its Comet browser, with classifiers, confirmation prompts and an external audit. These measures reduce the risk; they do not remove it, and no vendor promises complete protection.

On the development side, agent frameworks often ship useful building blocks: human approval steps, call tracing, guardrail integrations. But they do not make design decisions for you. Our AI agent frameworks comparison helps you pick a tool, and our Can I hand this to an agent? checker helps you assess the risk of a task before you automate it.

Pre-launch checklist

• Does the agent only have the permissions it needs, through dedicated and revocable credentials?
• Does it run in an isolated environment, network included?
• Is external content kept apart from instructions and sensitive tools?
• Are outputs checked by code before they are executed?
• Do irreversible actions require a clear human confirmation?
• Is every action logged, with an emergency stop ready?
• Have you run adversarial tests, and planned to repeat them after each update?

Check yourself Your support agent reads customer tickets and can issue refunds. Which measure reduces the risk the most?

Cap refunds inside the tool itself, and require human approval above a threshold. Tickets are untrusted content: anyone can write in them. An instruction like “never refund without a reason” can be overridden, whereas a limit coded into the tool applies whatever the model decides.