What Could a 2035 AI Failure Review Teach Us Today?

A short from my book, Boxing Einstein. Adapted from Chapter 10, “2035: The Report We Should Never Have to Write.” This is an adaptation, not a verbatim excerpt.

A fictional future review can help expose dependencies we might otherwise accept one decision at a time. It cannot tell us what will happen in 2035. The closing chapter of Boxing Einstein uses that date as a thought experiment: an imagined investigation asking how an institution could retain formal authority while losing the practical ability to exercise it (Clark, n.d., Chapter 10).

The commission in the chapter is fictional. So are its findings and the future sequence it investigates. There is no claim that a real commission has established these events, or that AI will inevitably take the path described. The device lets me examine a failure mechanism from the far end and then ask what evidence would distinguish preparation from reassurance today.

In the imagined account, no single decision announces a surrender of control. Successful systems make duplication look wasteful. Review becomes more automated. Alternative capabilities become harder to justify. Human signatures remain, but the organization gradually loses the information, practice and feasible choices that once gave those signatures substance.

Look for the chain of defensible decisions

The point isn't that efficiency is bad. The difficult case is that individual improvements can have defensible reasons while their combined effect deserves a separate review. A cheaper evidence pipeline, a consolidated supplier and a reduced fallback capability may each look reasonable in isolation. Together, in a hypothetical dependency chain, they could leave the institution with fewer ways to recognize and recover from a common failure.

I would use that possibility as a question, not announce it as a fact about every AI deployment. What independent capability does the organization expect to retain? Which recent changes affect it? If the primary system becomes unavailable or its account becomes unreliable, where would a different understanding of the situation come from?

A useful future-review exercise follows those dependencies into the present. It does not stop at an impressive description of catastrophe. The team should identify a condition that could be checked now: a fallback nobody has exercised, a record available only through the service being investigated or an escalation rule whose owner lacks the power to enforce it.

Ask what would disprove the story

A scenario becomes more useful when it admits evidence against itself. If the institution can reconstruct consequential actions independently, exercise a feasible alternative and demonstrate effective intervention under realistic constraints, the proposed loss-of-control story has less support. The exercise should recognize that result rather than insist on its dramatic ending.

Even working controls do not guarantee a good outcome. Independent reviewers can share a mistaken input. A fallback can be too slow for the actual decision window. An institution can act reasonably and still encounter harm. The purpose of the exercise is to make important assumptions and remaining exposure visible, not to promise that a checklist eliminates uncertainty.

I would end the exercise with a small number of owned decisions. Which dependency needs to change? What evidence would establish that the change worked? What limitation is being accepted, and when will it be reconsidered? Without that step, the fictional postmortem risks becoming another memorable presentation that leaves the operating arrangement untouched.

There is also a boundary to the method: an imagined outcome is not a probability estimate. The year 2035 supplies narrative distance, not a forecast deadline. Its value is whether looking backward from a possible failure changes a defensible decision before the failure exists.

That is why I finish the book with a report I don't want anyone to have to write. Read about Boxing Einstein for the full argument—and for the controls that make the fictional ending something to challenge rather than accept.