What Should an AI Decision Summary Leave In?

A short from my book, Boxing Einstein. Adapted from Chapter 7, “The Decision Is an Interface.” This is an adaptation, not a verbatim excerpt.

An AI decision summary should preserve the uncertainty, evidence and disagreement that could change the decision. Concision is useful; removing those elements can make a recommendation easier to approve while making it harder to evaluate. Chapter 7 of Boxing Einstein treats the decision interface as part of the control system, rather than a neutral screen placed in front of it (Clark, n.d.).

Imagine a hypothetical dashboard recommending a consequential operational change. The underlying analysis contains a disputed input, an alternative with different costs and a warning that the preferred option depends on a particular assumption. The executive summary presents a confidence score and a green recommendation. The summary may be concise and readable. The missing question is whether it preserves what the decision-maker needed to see.

I don't expect a senior leader to read every intermediate output. Summarization is necessary in many workflows. The design problem is deciding which information can be compressed without removing the reason a reasonable person might choose differently.

Keep the decision-changing uncertainty visible

I would organize the summary around the choice the reader must make. What action is proposed? Which consequence matters? What supports the recommendation? Which unresolved assumption could reverse it? That is more useful than filling a dashboard with a large number of indicators whose relationship to the choice remains unexplained.

Consider two uncertainties. One concerns an inconsequential detail in the supporting narrative. The other concerns whether the affected service can recover if the proposed change fails. Giving both the same visual weight creates noise. Removing both for brevity removes something more serious. The summary should distinguish uncertainty by what it could change, not merely by how easy it is to express numerically.

The reader also needs a route back to the evidence. A source label is helpful, but the reviewer should be able to distinguish an observation from an interpretation and an original record from a summary of another summary. Otherwise a polished interface can hide how far the recommendation has travelled from information somebody actually checked.

Authenticity doesn't settle the argument

In the book, I distinguish establishing the origin of information from establishing the truth of its contents. A message can come from the expected source while carrying an incorrect conclusion. Knowing that a recommendation was not altered after it was issued doesn't resolve whether the recommendation rests on a mistaken assumption.

That distinction changes what the interface needs to support. Provenance helps a reviewer understand where information came from. Corroboration asks what else supports the claim. The two questions complement each other; one cannot silently stand in for the other.

Time matters too. An evidence trail that can be reconstructed next month may support an investigation without helping today's decision. I would identify the checks that can be completed inside the relevant window, the important uncertainties that remain and who has authority to act with those uncertainties. A fast decision should not be described as fully verified merely because the interface had no room for the qualification.

The objection is that leaders need an answer, not a catalogue of doubts. I agree. A useful summary should make the recommendation clear and the consequential disagreement legible. It need not give every objection equal prominence. It should preserve the objection that could justify a different action.

The human may still choose the recommended course. What matters is that the interface has supported a decision rather than merely completed the last step of one already made elsewhere. Explore Boxing Einstein for the argument that control depends partly on what survives the journey from evidence to approval.