Before You Give an AI Agent Permission to Act

An AI system that answers a question can mislead you. An AI system that can open a ticket, change a configuration, retrieve private records, or send a message can do something considerably more expensive. The distinction isn't philosophical. It's an access-control decision, and the people approving the deployment need to understand exactly what authority they're handing over.

I'm interested in the point where a useful assistant becomes an operator. Once a model can call tools, its errors and manipulated instructions can cross from generated text into systems that other people depend on. NIST's AI Risk Management Framework addresses risks across the design, deployment, use, and evaluation of AI systems (Tabassi, 2023). Its generative-AI profile extends that discussion to risks particular to these technologies (Autio et al., 2024). Neither publication makes an autonomous system trustworthy merely because someone has completed a questionnaire. The real question is what the system can do when its judgment is wrong.

I would begin with a blunt inventory of capabilities. Can the agent read data, modify it, delete it, or transmit it? Can it change permissions, run code, approve payments, contact customers, or trigger an operational workflow? Which credentials does it use? Which systems trust those credentials? Who can withdraw them, and how quickly? The answer should be specific enough to survive an engineer asking to see the actual configuration.

This is familiar territory in security architecture. NIST's zero-trust guidance centers access decisions on resources and explicitly avoids granting implicit trust because of network location or ownership (Rose et al., 2020). An AI agent deserves no special exemption from that logic. It should receive the minimum authority required for a defined task, and that authority should be constrained by the action, resource, context, and duration. If an agent only needs to summarize an incident ticket, it does not need permission to close the ticket or change the firewall.

An especially bad design gives an agent a broad service account because integrating fine-grained permissions would delay launch. The agent may appear to perform one harmless workflow, but its credential can reach far beyond the visible interface. That is not an intelligence problem. It's an ordinary privilege problem with a novel and unpredictable decision-maker sitting in front of it.

A system prompt can tell a model not to reveal secrets or perform unauthorized actions. That instruction may be useful as part of the application design, but it is not equivalent to an enforcement mechanism. A model consumes instructions and content in forms that can be difficult to separate perfectly. An attacker may place hostile directions in a retrieved document, web page, repository comment, or other lower-trust material the agent is expected to read. The danger is that content intended as evidence gets treated as authority.

NIST's generative-AI profile identifies risks associated with generative systems and encourages risk management throughout their lifecycle (Autio et al., 2024). The Secure Software Development Framework supplies a complementary discipline: build security into software design, development, and maintenance rather than attaching it after deployment (Souppaya et al., 2022). My operational translation is straightforward: assume the model can be persuaded to request an action it should not perform, then make sure the tool refuses that action anyway.

A read-only document summarizer should have read-only access enforced by the document service. An agent allowed to draft a customer email should not automatically be allowed to send it. A deployment assistant may prepare a change, but production execution can require a separate identity, policy check, and approval. These boundaries belong in the systems performing the actions, not only in prose addressed to the model.

"Human in the loop" sounds reassuring until you examine what the person is being asked to approve. If the interface displays a vague summary and a green button, the human may be approving a set of actions they cannot inspect. If the system asks for hundreds of approvals each day, people will eventually click through them. Oversight becomes ceremonial when the operator lacks the time, evidence, or authority to intervene.