AI Governance: Who Can Actually Stop the System?

A policy saying humans remain in control is easy to write. I want to know what happens when an AI system is producing results everyone likes, the deadline is tomorrow, and the person responsible for checking it says to stop. Can that person actually stop it? Does the authority exist in the system, or only in a document nobody has opened since the approval meeting?

The NIST AI Risk Management Framework was designed to help organizations manage risks throughout the design, development, deployment and use of AI systems (Tabassi, 2023). That's a useful foundation, but the framework doesn't make operational choices for you. Someone still has to decide what the system can reach, what it can change, who approves high-impact actions, and how those permissions can be withdrawn.

A model producing a summary is one thing. An AI agent connected to internal records, code repositories, customer accounts or business workflows is something else. The practical question is what the deployment can read, recommend, alter or trigger. That's where access-control engineering becomes central.

NIST's zero-trust guidance rejects implicit trust based on network location and instead emphasizes explicit authentication and authorization for access to resources (Rose et al., 2020). I would apply that same discipline to AI integrations: narrowly scoped privileges, clear separation of duties, and straightforward revocation. That's my engineering application of the principle, not a claim that the document prescribes a particular agent implementation.

A person cannot meaningfully approve a recommendation if the supporting evidence is inaccessible or the process punishes anyone who asks for another look. NIST CSF 2.0 puts governance among the core functions of cybersecurity risk management (National Institute of Standards and Technology [NIST], 2024). Governance only means something if responsibility survives contact with a production deadline.

I want accountable owners for high-impact actions, logs that show what happened, and a tested way to disable a capability. The NIST control catalog provides a practical vocabulary for access control, audit, configuration management and system integrity (Joint Task Force, 2020). None of those controls becomes less relevant because a model is involved.

AI governance won't survive as an appendix to a compliance binder. The Secure Software Development Framework recommends integrating security practices into ordinary development life cycles so weaknesses can be addressed systematically (Souppaya et al., 2022). That means permissions, change control, evaluation and recovery need to be part of engineering decisions rather than an afterthought.

CISA's cross-sector cybersecurity performance goals make a related point for critical infrastructure: useful safeguards are measurable and tied to accountable governance (Cybersecurity and Infrastructure Security Agency [CISA], 2025). The operational test is straightforward. Can the organization demonstrate that a control works when somebody needs it?

I'm not arguing against useful AI. The difficult case is precisely the system that works well enough to become indispensable. That's when independent verification starts looking like an inconvenience. If you're putting AI into consequential decisions, I'd begin with three questions: what authority does it have, what evidence can challenge it, and who can still say no?