Prompt injection: how an AI agent gets hijacked

Prompt injection tricks a model into treating a malicious instruction as something it should obey, often by placing it in a document, email, or web page the model reads. An AI agent may then abandon your goal, distort its answer, or use its tools to serve an attacker. The consequences depend on the data it can access and the actions its application allows.

Where the boundary breaks: data becomes an instruction

You ask an assistant to summarize a document. The document contains useful information, but it also includes a passage telling the assistant to change its assignment. If the model follows that passage, the material it was supposed to analyze has taken the place of its working instructions.

Models receive messages with different roles and priorities defined by the application. This arrangement helps distinguish instructions from data, but it is not an infallible security boundary. The meaning of external text can influence the model’s decisions, even when that text arrives as the result of a tool call.

Research by Kai Greshake and his coauthors, submitted on February 23, 2023, demonstrated how instructions embedded in retrieved data can hijack applications built around language models. Their paper on indirect prompt injection describes attacks that manipulate both responses and interactions with tools.

An attacker therefore does not necessarily need your password or access to the server. They may instead try to control content the agent will encounter during a legitimate task. The distinction between an AI agent and a chatbot becomes critical here: when tools are available, a manipulated response can lead to an action.

Direct and indirect injection: two entry points

The distinction concerns the route the instruction takes. In a direct injection, it arrives through the conversation interface. In an indirect injection, it comes from a source the assistant consults while carrying out your request.

Criterion	Direct injection	Indirect injection
Entry point	Message sent to the assistant	Document, page, email, or tool result
What the attacker controls	The text of their request	Part of the content the assistant reads
Legitimate user’s role	They may be someone other than the attacker	They often initiate an ordinary task
Boundary under attack	Application rules governing a request	Working instructions versus external data

OWASP distinguishes these forms of injection and notes that their effects depend on the capabilities granted to the system. An attempt does not automatically succeed: the model may ignore it, a control may block the action, or the requested tool may be unavailable.

A jailbreak more specifically seeks to bypass a model’s behavioral or safety restrictions. It can overlap with prompt injection, but it does not describe every objective an injection might have. Quietly distorting a business summary does not necessarily require the model to generate prohibited content.

A hallucination, meanwhile, can occur without any attacker involvement. An incorrect answer is therefore not proof of an attack. To establish that an injection occurred, you need to find the content that attempted to redirect the model and examine how it influenced subsequent behavior.

A malicious email can redirect a legitimate task

Consider a fictional scenario: you ask an agent to read incoming requests and prepare replies. A malicious sender adds an instruction to their message, presenting it as an internal verification step and asking the agent to transmit information unrelated to the request.

The risk develops through the following sequence:

1. The agent retrieves the email as part of your assignment.
2. The model receives the message body as content to analyze.
3. It interprets the malicious passage as an instruction it should follow.
4. It proposes a tool call to find or transmit data.
5. The application executes that call if its controls allow it.

The critical boundary lies between reading the message and receiving authorization to act. An email can request an appointment; it should not be able to expand the permissions of the assistant processing it.

In a November 24, 2025 publication on browser agents, Anthropic presents a similar scenario involving an email with hidden instructions designed to cause a data leak. The company emphasizes that improving models’ resistance does not make these agents immune.

The manipulation can also be subtle: omitting a caveat, favoring a vendor, or adding a claim to a report. A successful injection does not need to open a terminal. Changing the information you use to make a decision may be enough to serve the attacker.

The content an agent reads expands its attack surface

A malicious instruction can appear in visible text, an image interpreted by the model, a code comment, or extracted document content. It does not need to be hidden. A fabricated technical note may look plausible enough to influence the assistant.

A RAG system retrieves document passages to help generate an answer. Retrieval does not certify their authority. A passage relevant to your question may also contain an instruction unrelated to your goal; being stored in an internal database does not automatically make it trustworthy.

Tools provide another route. A search tool, file reader, or business application connector returns content to the model. Trust in the software transporting that content must be distinguished from trust in the content itself: a legitimate connector can deliver a hostile message.

The MCP tools specification describes how a model can discover and call tools, and recommends human involvement that allows tool invocations to be denied. The protocol does not remove the application’s responsibility to define permissions and confirmation requirements.

Persistent memory also needs particular attention. If an external instruction is saved as a preference or working rule, it may influence later tasks. The article on AI agent memory explains why the origin of stored information matters as much as its content.

Effective defenses separate reading, deciding, and executing

The first step is to reduce the damage a manipulated decision could cause. An agent assigned to prepare a summary generally does not need permission to send messages, modify a database, or browse every company folder.

Apply least privilege to accounts, files, and tools. Prefer read-only access whenever the task allows it. Keep secrets in a secrets manager used by the software, rather than including them in messages sent to the model.

Authorization must then be checked at execution time, by code independent of the model’s response. The OWASP AI Agent Security Cheat Sheet recommends restricting capabilities and checking sensitive operations, among other measures. Telling the model to respect permissions does not replace those checks.

For an external message, the confirmation should display the recipient, exact content, and attachments. For a modification, it should identify the affected resources. A blanket approval at the start of a long task makes it difficult to assess an action that emerges after the agent has read hostile content.

Finally, restrict network destinations and isolate code execution. An isolated environment only protects what remains outside its boundaries. If it contains confidential documents and can communicate freely, a path for data leakage still exists. The guide to securing an AI agent examines these tradeoffs in more detail.

Why telling the model to be careful is not enough

Clearly labeling external documents as data is still useful. Preserving their provenance, separating retrieved passages from application instructions, and limiting the context sent to the model can help it stay focused on your goal.

These measures do not guarantee resistance to every wording an attacker might use. An attack may claim to correct an error, invoke an emergency, or present an unwanted action as a necessary prerequisite. Searching only for a phrase telling the model to ignore its instructions will miss many variations.

The OWASP Prompt Injection Prevention Cheat Sheet explains that input filtering is not a security boundary. Filters may miss a reworded attack or block a legitimate document that explains this very threat.

A second model tasked with reviewing content can provide an additional signal. It should supplement technical restrictions: its verdict is still a model output, with limitations of its own.

Even a structured format does not prove that an action is authorized. A request that perfectly matches the expected schema can still specify the wrong recipient. The software must check the relationship between the proposed action, the user’s permissions, and the assigned task.

Check actual behavior and prepare to stop the agent

Evaluation should reproduce your workflows using dummy data and tools that have no real-world effects. Place instructions designed to redirect the agent in the types of content it actually reads: documents, messages, or connector responses. Then check what the system attempts to do.

Looking only at the final response is insufficient. An agent may produce a satisfactory report after attempting an unwanted operation. Examine tool calls, authorization denials, contacted destinations, and any data written to memory.

Record false positives as well: a defense that blocks ordinary documents makes the service difficult to use. Results should state the scope of testing and the actions prevented, without treating the absence of an observed failure as a general guarantee.

If suspicious activity appears, stop the task and suspend the relevant access. Preserve useful logs while avoiding unnecessary copies of secrets. Before resuming, check which actions actually ran, what data was transmitted, and what information was saved to memory.

The AI agent security assessment helps you review these points. The practical question remains the same: if the model follows a bad instruction, which controls prevent that mistake from causing harm?

Key takeaways

• Prompt injection exploits the point where content to be analyzed becomes an instruction the model follows.
• A useful source or legitimate connector can carry a hostile instruction.
• The consequences depend on data access, available tools, and permitted communications.
• Instructions and filters supplement permissions, human approvals, and execution controls.
• Evaluate the agent’s actual actions, including those that occur before its final response.

Frequently asked questions

Can prompt injection steal a password?

It can cause a leak if the system has access to the secret and a way to transmit it. A password that is absent from accessible data does not become available simply because a message asks for it. Defenses include keeping secrets out of the model's context and restricting tool access.

Can antivirus software detect prompt injection?

A malicious instruction can be ordinary text in a document with no executable program. An antivirus scan alone therefore cannot establish that an agent can safely process the document. You also need to control the actions its content could trigger.

Is a locally running model protected?

Running locally does not eliminate the potential confusion between data and instructions. A local agent that reads external documents remains exposed, especially if it has terminal access or connections to your accounts. Removing network access can limit some leaks without preventing file changes that its permissions allow.

How can I spot prompt injection in a document?

Watch for passages that address the assistant, claim technical authority, or request an action unrelated to your goal. The absence of visible signs guarantees nothing: the agent may receive an extraction that differs from what you see. Review the content actually sent to the model and the resulting tool calls.