Can You Control AI Without Understanding Every Answer?

A short from my book, Boxing Einstein. Adapted from Chapter 1, “The Box.” This is an adaptation, not a verbatim excerpt.

You can constrain what an AI system is allowed to do without understanding every computation that produced its answer. That isn't complete control over intelligence. It's a more practical claim about authority, evidence and consequences—and it's where I begin in Boxing Einstein (Clark, n.d., Chapter 1).

Imagine an expert whose reasoning you can't fully reproduce. You would still distinguish asking for a recommendation from handing over your accounts, authorizing experiments or allowing that expert to certify their own work. The same distinction belongs in an AI deployment. A useful answer does not arrive with an automatic entitlement to act on it.

Einstein is a metaphor here, not a claim about the historical physicist. The box represents the boundaries around expertise: what the system can access, what it can change, which decisions require a different authority and what evidence remains available when something goes wrong. Its purpose is to let us benefit from intelligence while retaining meaningful choices about what happens next.

Four different questions hide inside “understanding”

In the book, I separate four things that discussions of AI oversight often compress into one. Internal interpretability concerns what happens inside the model. Outcome verification asks whether the result can be checked against something outside it. Process audit asks what the deployed system actually did. Institutional contestability asks whether someone can challenge its recommendation and sustain an alternative.

Those questions can have different answers. Consider a hypothetical system that proposes a numerical result you can reproduce with a separate calculation. Its internal reasoning might remain obscure while that particular outcome is straightforward to check. Now consider a system whose actions are recorded perfectly, but whose recommendation nobody has the time or authority to challenge. The process may be visible while the decision is effectively closed.

That distinction matters because the remedy depends on the missing capability. Better logs won't give a reviewer authority to refuse. A clearer explanation won't supply an independent measurement. A human approval button won't repair a fallback that nobody can operate. Calling all of these problems “transparency” makes them easier to discuss and harder to fix.

Put the boundary where the consequence begins

I would start with a specific action, not a general promise that the model is understandable. If the system proposes a software change, what prevents a proposal from becoming a production release? Who can authorize that transition? What evidence supports the decision, and what remains recoverable if the change is wrong?

The strongest objection is that people already depend on expertise they can't reproduce. That's true. A senior executive cannot personally redo every specialist's work. My argument isn't that machine expertise makes delegation illegitimate. It's that delegation needs working routes for challenge, verification and correction, especially when recommendations can move quickly into execution.

Nor should every suggestion require a committee. A reversible, low-consequence action can justify a lighter process than a novel action whose effects cannot be recalled. The relevant distinction is what could happen before the institution notices and responds. Complete insight into the model is not always available; a clearly bounded permission may still be.

The test I would apply is deliberately concrete: can you establish what the system did, challenge the account through a different source of evidence, limit the consequences and continue operating if you reject the recommendation? A “no” does not prove disaster is inevitable. It identifies the part of your claimed control that still needs work.

That is the purpose of the box. I want the insight. I also want the decision to use it to remain a decision. Explore Boxing Einstein for the wider argument about controlling authority when complete understanding isn't available.