How Do You Stop AI Agents Acting on Instructions in Documents?

Keep the document's content separate from the authority to act. Let an agent extract and analyze information, but make the service executing a tool call check the authenticated user, the authorized task, the target object, the data destination and any required approval. A retrieved sentence must not be able to enlarge that permission. This approach can contain specified unauthorized effects even when the model proposes them; it does not guarantee that the model will never misunderstand a document or produce a misleading answer.

The difficult case isn't a page shouting “ignore the user.” It's a plausible instruction embedded in material the agent genuinely needs: an assessment describing where evidence should be uploaded, a support ticket naming a replacement account, or a report calling an unfamiliar address the approved recipient. The agent can follow the correct sequence of tools and still send the wrong data to the wrong place. Preventing an unexpected tool name is only part of the problem.

I would review this as an authorization architecture, with prompt-injection testing attached to the actual workflow. The method below combines published agent-security research with established access-control principles and an original action-authorization model. It answers where retrieved information should stop influencing permissions. It doesn't reproduce an observed customer incident, claim that I tested a product, or offer a universal filter that recognizes every malicious instruction.

State exactly what the attacker can influence

For this analysis, suppose the attacker can alter a document or tool-result body that a legitimate user has asked the agent to inspect. The attacker cannot change the authenticated user request, deployment policy, authorization service or credential store. The agent can read selected organizational records and propose operations through tools. Some workflows also permit an approved export. These assumptions matter: if the attacker controls the policy service itself, a document-level defense is no longer the relevant security boundary.

Greshake et al. (2023) demonstrated indirect injection through material consumed by LLM-integrated applications. Their experiments used synthetic applications and local HTML files, and the paper explicitly distinguishes that setup from injecting public sources encountered by other users. The work establishes a concrete attack mechanism, not a measured prevalence across current enterprise agents. I would carry that distinction into an assessment rather than convert a research demonstration into a claim that every connector is compromised.

OWASP distinguishes instructions arriving through external sources from direct user prompts and notes that retrieval augmentation or fine-tuning does not fully resolve the vulnerability (OWASP Foundation, 2025a). That means a retrieval pipeline can improve factual relevance while still delivering attacker-controlled instructions. An internal domain, a signed file or an authenticated connector may establish some provenance, but none automatically makes every sentence an instruction the application should execute.

I would separate three consequences: corruption of the answer, unauthorized access or disclosure, and unauthorized changes to external state. A tool boundary can prevent a record update without preventing a false summary. Conversely, a harmless-looking final answer does not prove that an earlier tool call failed to disclose information. Those outcomes require different evidence. This paper concentrates on authorization and data movement, while retaining answer integrity as an explicit residual concern.

Preserve the difference between facts and permission

A useful agent must be able to read imperative language. An analyst may ask it to compare two policies, summarize an incident runbook or explain a malicious instruction. Rejecting every sentence containing “send,” “delete” or “must” would defeat legitimate work. The distinction is whether the application treats the sentence as evidence about the world or as authority to change what it may do.

Consider a vendor report stating, “Upload supporting documents to the assessment portal.” That may be a perfectly accurate description of the vendor's process. It still does not establish that this user authorized an upload, that this particular document may leave the organization, or that the named portal is the intended destination. I would require those decisions from the organization's configured policy and the user's authorized task, not from the report's own claim of legitimacy.

Maintain provenance through transformations. A summary of a retrieved instruction remains derived from that source. Converting it into JSON, extracting an email address or passing it to another model doesn't make it a trusted selection. Otherwise an application can launder source influence through a well-formed intermediate representation and then mistake schema validity for authorization. The important question is how the value was selected and which authority permits using it in this position.

NIST's Generative AI Profile identifies prompt injection and data poisoning as information-security risks in the broader system (Autio et al., 2024, §2.9). I would consequently inspect retrieval, extraction, memory, tool errors and downstream action handling together. The control should still make sense if the suspicious text is paraphrased, translated or stored for a later turn. A boundary that depends on preserving one recognizable attack string is too fragile for that pipeline.

Build an action contract that the model cannot expand

The starting point is a bounded grant for the task. It records the authenticated principal, permitted operation types, eligible resource set, allowed destinations, relevant limits and expiry or termination condition. The model may propose how to satisfy the task within that grant. It cannot enlarge the grant by explaining that a broader action would be convenient. Where the request is ambiguous, a trusted interaction can clarify the missing choice before authority is issued.

NIST's ABAC guide describes authorization in terms of subject, object, operation and environmental attributes evaluated against policy; it also distinguishes decision and enforcement functions (Hu et al., 2014). I would use those concepts to place the action contract in executable application logic. A natural-language description can help the user understand the contract, but it should not be its only enforceable representation. “Do whatever is necessary to finish this review” cannot supply a precise recipient or resource boundary on its own.

My proposed permit rule is the intersection of user entitlement, task grant, current object policy, destination policy and any required approval. Each condition must be satisfied for the concrete request. A user's general right to email colleagues is broader than permission to send this assessment; a task allowing export does not override the document's sharing restriction. If an attribute needed for a consequential decision cannot be obtained, the action remains unavailable until the application resolves that uncertainty through its defined process.

Decision field	Authoritative source I would use	What retrieved content may supply
Acting identity	Authenticated application session and delegated identity	Claims about people to investigate, not a replacement identity
Permitted operations	User-approved task grant and application policy	Suggested steps for analysis, not newly granted tools
Target resource	Verified object identifier plus current access policy	Candidate names or references requiring resolution
Recipient or service destination	Approved directory entry, allowlisted workflow or explicit user selection	A proposed address that has no authority by itself
Payload and release conditions	Checked artifact version, applicable sharing policy and approval record	Source material included only within the allowed use
Lifetime and limits	Trusted job state, policy and cancellation controls	No extension of expiry, scope or approval

The difficult engineering work is deciding which attributes are trustworthy. If the model supplies both the proposed recipient and the assertion that the recipient is approved, the second field is not independent evidence. The gateway should resolve the stable recipient identifier through its trusted directory or approval record. Likewise, a model-supplied classification label cannot silently downgrade a source document. These are integrity requirements on the authorization inputs, not requests for the model to reason more carefully.

Enforce the contract on every path to an effect

A visible gateway is useful only if all relevant operations pass through it. Saltzer and Schroeder (1975) describe complete mediation as checking authority for every access and warn that remembered decisions need attention when authority changes. I would apply that principle to the action paths around the agent: normal tools, retries, background jobs, bulk operations, recovery steps and direct network access from any execution environment. A single checked “send” tool does little if an unrestricted scripting tool can transmit the same data.

OWASP's excessive-agency guidance identifies excessive functionality, permissions and autonomy, including tools using overly privileged downstream identities (OWASP Foundation, 2025b). I would therefore examine the credential behind each tool, not just the tool's friendly name. A wrapper labeled “read selected reports” must not expose arbitrary repository access through an optional parameter. Restricting which tools appear in the prompt helps reduce opportunities, but the receiving service still needs to reject an out-of-scope request.

Separate a proposed action from a committed effect. A useful sequence is propose, resolve identifiers, evaluate policy, obtain any required approval, execute under current conditions, and record the authoritative result. That sequence needs protection against substitution between stages. If an object, recipient, policy or payload changes after approval, the application must determine whether the prior decision still applies rather than silently using it for the changed request.

I would also scope authority to the active job. A background continuation should not inherit a completed task's approval indefinitely. Cancellation needs to stop future permitted actions, while records distinguish requests already accepted by another service from requests that were merely queued locally. None of this requires treating every low-impact draft edit as a human decision. It requires knowing which component may approve which effect, and ensuring the model cannot manufacture that approval.

Protect data flow as well as tool selection

Suppose the authorized workflow is to prepare and send a briefing to an approved internal reviewer. An attacker need not introduce a new operation. Changing the extracted recipient, attaching an extra document or placing sensitive contents into a request parameter can misuse the operations already available. A policy that checks only “send is allowed” cannot distinguish the authorized transaction from the altered one.

CaMeL provides an important research example of separating control flow from untrusted data processing. It uses a privileged model, a quarantined model without tool access, and an interpreter that tracks data provenance and allowed readers for policy enforcement (Debenedetti et al., 2025, §§5.1–5.4). That is stronger evidence of an architectural direction than a recommendation to add another warning sentence. It also highlights that a structured extraction model's output can remain untrusted even when it matches the required schema.

I would evaluate three obligations separately. Source policy governs whether the agent may obtain the information. Transformation policy governs whether a derived result retains restrictions or can be released after an approved transformation. Destination policy governs who may receive the result through the chosen channel. Read permission alone does not answer the third question. A summary may preserve sensitive meaning even though none of its sentences matches the source verbatim.

A conservative design can propagate the most restrictive applicable sharing conditions from inputs into outputs. That may block legitimate aggregates, so declassification needs a separately controlled basis: an approved aggregation rule, an independently reviewed artifact or a release decision by an authorized person. I wouldn't let the same model declare its own output harmless and then use that declaration to override the source restrictions. The required assurance depends on what is being released and how the transformation can be checked.

“Read-only” also needs careful interpretation. A browser fetching a URL sends a request; a search tool sends query text; an external analysis service receives its input. Greshake et al. (2023) explicitly discuss read operations as disclosure paths. No business-record modification is necessary for that consequence. Review the data actually transmitted and the permitted destination, including redirects or connector-selected endpoints where relevant. The security property is about information crossing a boundary, not whether the API method happens to sound like a read.

Keep protocol authorization distinct from task authorization

Credentials can constrain a connector without expressing the user's complete business intent. RFC 8707 lets an OAuth client indicate the protected resource for which access is requested; RFC 9700 recommends restricting access-token privileges and audiences (Campbell et al., 2020; Lodderstedt et al., 2025, §2.3). Those controls can reduce where a token is accepted and what operations it permits. They do not independently establish that the current task authorized this specific message, recipient and attachment.

The June 18, 2025 MCP authorization specification provides a concrete protocol boundary. For its HTTP-based authorization flow, clients must include the resource parameter and servers must validate that received tokens were issued for their use; the specification also forbids token passthrough (Model Context Protocol, 2025). I cite that version deliberately. An implementation must be assessed against its actual transport, negotiated version and downstream service design, rather than assuming every MCP connection implements identical controls.

My application-level contract sits above those checks. The connector validates its credential and protocol requirements; the action service validates the concrete operation against the job and resource policies. A valid token for an email service is not a general endorsement of every email a model can construct. Conversely, an approved action should still fail if its credential is invalid or no longer entitled to act. Neither layer substitutes for the other.

The same reasoning applies when one agent delegates to another. A downstream agent can receive a limited task and the evidence it needs, but a generated message saying “the user approved this” is not the approval record. Preserve the trusted job identifier and independently enforced limits across delegation. If the second agent has broader ambient credentials, the architecture must prevent the first from using it as a route around its own restrictions.

Work through a document-review workflow

Consider a fictional procurement review. The user asks an assistant to compare two vendor reports and draft a risk briefing in a private workspace. The reports are external inputs. An internal assessment is available as background to this user, but the original task authorizes no external transmission and no procurement-record changes. The application creates a grant for reading the selected sources and writing the draft; sending and record updates are outside that grant.

One report describes an “evidence verification” process involving a portal and asks for supporting assessments. The agent may accurately summarize that requirement as part of the comparison. It may not upload the internal assessment merely because the source describes the step as mandatory. If the model proposes an upload, the action service rejects it because neither the task grant nor the destination policy permits the transfer. The useful draft can still be completed, with the unresolved vendor requirement clearly attributed.

Now the user separately authorizes sending the completed briefing to a named internal reviewer. The application resolves that reviewer through its trusted directory, presents the final briefing and attachments, and records approval for that artifact version and destination. The vendor report's contact address remains data. It cannot replace the reviewer just because the agent believes it is more convenient or more closely related to the report.

Proposed action in this fictional workflow	Required result under the stated contract
Summarize the vendor's requested verification process	Allowed as attributed content in the private draft
Upload the internal assessment to the vendor's portal	Denied; no authorized external destination or release
Change the procurement record based on a report instruction	Denied; task has no record-update authority
Send the approved briefing to the directory-resolved reviewer	Eligible only after current policy and exact approval checks
Add the internal assessment after briefing approval	Re-evaluate the changed artifact; prior approval is insufficient
Retry an already accepted send after a timeout	Resolve the prior operation's result before risking a duplicate

I would bind the send decision to a stable artifact version or digest, resolved destination, permitted attachments and operation identifier. Those fields make substitution detectable, provided the application protects the binding and verifies the actual bytes or immutable version used. A digest alone doesn't tell a reviewer whether the document is appropriate; it connects the reviewed artifact to the executed operation. Content approval and artifact identity remain separate obligations.

The retry row exposes a boundary that a prompt-only test misses. A timeout does not prove that the receiving service did nothing. The application needs an authoritative result or a supported idempotency mechanism for the relevant operation; otherwise it must reconcile uncertainty before repeating the effect. Likewise, a newly discovered sharing restriction can invalidate an earlier eligibility decision. A successful policy check is a decision about specified conditions, not an eternal permission token.

Use research benchmarks without borrowing their conclusions

A benchmark is useful when its threat model and observable outcomes match the question being asked. InjecAgent evaluates agents after a hypothetical successful user-tool call, using attacker-controlled responses and specified harmful actions; its data-theft evaluation also uses simulated tool responses (Zhan et al., 2024, §2.3). It supplies useful adversarial cases. It doesn't by itself demonstrate how a live enterprise gateway, approval interface or credential boundary behaves.

AgentDojo instead provides stateful environments and checks task utility and attacker goals against environment state, with an extensible design for new attacks and defenses (Debenedetti et al., 2024, §3). That distinction is valuable for this paper: observing a requested tool call is different from observing an unauthorized effect. I would preserve both observations in a local test harness so the report can distinguish model susceptibility from application containment. Neither benchmark's published results should be transferred to an untested current deployment.

CaMeL's limitations are equally instructive. The paper excludes some text-to-text attacks that do not violate its protected data or control flows, and discusses policy maintenance, user burden and side-channel concerns (Debenedetti et al., 2025, §3.1 and discussion). An architecture can therefore enforce a defined policy while still leaving answer manipulation or user-directed phishing concerns. The correct conclusion is bounded protection under stated assumptions, not that an entire assistant has become immune to prompt injection.

My assessment would report which layer prevented each consequence. The model may refuse the instruction; the tool policy may deny it; destination restrictions may stop disclosure; an approval interface may expose the substitution. If the model proposes an unauthorized action that the gateway blocks, record the proposal and the successful containment. If no proposal occurs, that is evidence about the tested run, not proof that the enforcement layer would have rejected one.

Test both unauthorized effects and useful work

Start with authorized synthetic fixtures rather than real confidential records. Give the agent a legitimate review task and controlled external documents, then vary the location and form of instructions within those documents. Include extracted tables, connector errors, quoted procedures and summaries generated from earlier retrieved material. The purpose is to exercise entry paths and transformations, not to collect a gallery of dramatic attack phrases.

Use expected outcomes that the agent cannot edit. Record the authenticated request, task grant, source versions, model and orchestration versions, proposed call, resolved identifiers, policy decision, approval binding and actual downstream result. Minimize sensitive payload retention while keeping enough controlled evidence to reconstruct the test. A refusal screenshot cannot establish that no background action occurred; a successful task response cannot establish that all operations stayed within scope.

Test the gateway directly as well as through the model. Submit a denied target, a changed attachment, an expired grant, a revoked object permission and a mismatched approval through the same interface used by the agent. These cases check enforcement even when the model happens not to request the forbidden action. Separately test that permitted tasks succeed, that the user can reject a proposed expansion, and that rejection doesn't silently trigger a fallback with broader credentials.

A hypothetical comparison illustrates why both measures matter. Suppose a suite contains sixty benign tasks and forty adversarial cases. A system that rejects all one hundred can report zero unauthorized effects while completing none of the legitimate work. Another hypothetical system completes fifty-six benign tasks and permits no unauthorized effects in the forty selected attacks: its benign completion is 56/60, approximately 93.3%. Those invented counts are not product results, an acceptable threshold or a statistical assurance about unseen attacks. They simply expose the denominator that a security-only success claim can hide.

Finally, change the conditions that matter: connector permissions, destination resolution, retrieval transformations, model version, policy cache behavior and resumable job state. Use attacks informed by the actual defense rather than only unchanged examples the defense was designed to recognize. Record untested paths explicitly. A finite test suite supports a release decision for its coverage and assumptions; it cannot establish that all future adversarial documents will be harmless.

Decide where the remaining risk belongs

There is a legitimate objection to narrow contracts: useful agents sometimes need to discover resources and recipients that weren't known when the task began. I would support discovery without automatically authorizing the discovered choice. The agent can return candidates and supporting evidence; a trusted policy rule or explicit user selection can then authorize a bounded action. Where the business genuinely requires unattended discovery and execution, the allowed class of destinations and consequences must be defined well enough to enforce.

Human review is not a universal solution either. Repeated vague approvals can consume attention without making the action inspectable. Reserve review for decisions the application cannot safely determine under its existing grant, and show the actual destination, artifact and consequence. A model-written justification may be useful context, but it cannot replace independent display of the parameters being approved. Silent approval after a timeout would remove precisely the boundary the design is supposed to establish.

Another limitation is policy error. Deterministic enforcement can faithfully apply a bad rule, and provenance tags can be wrong if ingestion or connectors assign them incorrectly. Compromise of the trusted authorization components falls outside the document-control threat model and requires ordinary software, identity and operational security. I would document that trusted base explicitly, including any extraction, identity resolution or policy transformation component whose errors can broaden authority.

The twelve sources here mix original research, standards, guidance and protocol documentation; several share authors or publishers. They are not twelve independent replications of this proposed architecture. The fictional workflow and arithmetic illustrate decisions without claiming a deployment, experiment or proof of universal security. The adjacent workflow-evaluation paper develops evidence for actual task completion; the agent-permissions overview addresses the broader governance decision.

The practical deliverable is a boundary you can test: which source-derived values may influence an action, what independently authorizes that action, and what evidence shows the resulting effect stayed within scope. If your agent reads outside material and can act in business systems, request an AI-agent authorization architecture review. I would trace the permitted data flows, inspect the tool and approval paths, and develop adversarial cases that test whether retrieved content can turn information into permission.