What to ask an AI vendor before your data goes in

Procurement questionnaires for AI products tend to copy the ones for SaaS, and they miss the questions that matter.

The questions I'd add

Where does my data go, and does it come back? Prompts, documents, and outputs: are they retained, for how long, and are they used to train anything? "We don't train on your data" is a start. Ask whether that applies to the sub-processors too.

What can the system do, not just say? If it only answers questions, the blast radius is leakage. If it has tools, connectors, or the ability to act on your behalf, the blast radius is everything those tools reach. Ask for the list.

How do you handle prompt injection? Not whether; how. A vendor who says it's solved is either wrong or selling something else. A vendor who describes the boundary between instructions and data, what the system refuses, and how you'd find out it went wrong is being honest.

What evidence do I get? Logs of what was asked, what was retrieved, what tools were called, and what was returned. If you can't reconstruct what the system did, you can't investigate it, and you can't defend a decision made with it.

How do I turn it off? Per user, per connector, and entirely. Ask how long each takes.

Why this matters more than the usual checklist

A conventional SaaS tool does what its menu says. An AI system does what it's asked, by anyone who can get text in front of it, including attackers who put that text in a document your staff later upload. The vendor's security posture matters, but the system's permissions matter more. That's the same argument I make in Boxing Einstein, applied to a purchase order.

If you're mid-evaluation and the answers you're getting are vague, that's worth a second opinion before the contract is signed.

Explore: More on AI security · Work with me