The first question in an enterprise security review is some version of "where does our text go". It sounds simple. Most teams answer it with the name of one provider, and most of those answers are incomplete, not because anyone is hiding anything, but because nobody traced the path.
This is how to trace it, and what each hop obliges you to be able to say.
The hops, in order
For a typical integration the text crosses more boundaries than the architecture diagram suggests:
- Your application. Where the prompt is assembled, and where it may be logged before anything else sees it.
- Your logs and error tracking. The hop teams forget. An exception handler that serialises the request body has just copied a prompt into whatever SaaS receives your stack traces.
- Your gateway or proxy, if any. Its own logs, its own retention, its own region.
- The model provider. Named in the answer everyone gives.
- The provider's own sub-processors. The hop nobody asks about, and the one an auditor will.
- Anything reading the response on the way back. Evaluation harnesses, moderation services, analytics that capture output text.
A complete answer names every hop, says what is retained at each, and says for how long. If you cannot fill that table today, that is the finding, not the review.
Trace it, do not diagram it
Architecture diagrams describe intent. Find the actual path:
Grep for what logs the request. Any line that logs a request body, an exception with context, or a "debug" dump of an outgoing call. Include your framework's defaults — several log request bodies on error without being asked.
Read the SDK's own logging settings. Provider SDKs frequently have a debug mode that prints full request and response. If it can be turned on by an environment variable, assume it has been on somewhere at some point.
Check what your observability vendor captures. Traces with attributes, error payloads, session replay. Session replay is the sharpest edge: if a user typed the prompt into a text field, replay may have recorded the keystrokes, and that store is usually governed by nobody's data policy in particular.
Ask the provider for its sub-processor list. It is a published document at every serious vendor. If asking feels like an unusual request, that is itself information.
Three answers worth getting in writing
1. Is content used for training? Ask specifically about the tier you are on — the answer often differs between consumer, standard API and enterprise agreements at the same vendor. Get the sentence, not the marketing page.
2. How long is content retained, and where? Most providers retain inputs and outputs for some abuse-monitoring window even when they do not train on them. That window is the number your auditor wants, and it is rarely on the front page.
3. What is kept when you delete? Deletion usually means the primary store. Backups, audit logs and abuse-monitoring copies have their own clocks. "Deleted" and "gone" are different words.
The distinction that makes this tractable
Separate content from operational metadata and the conversation gets much easier.
Content is the prompt and the completion — the highest-risk data in the system, needed only to serve the request. Operational metadata is token counts, latency, model, error code, request id — needed to bill, to debug, and to notice a provider degrading, and carrying almost none of the risk.
A system that keeps metadata indefinitely and content not at all is easy to explain and easy to defend. One that keeps "logs" without distinguishing the two is neither, and that is the shape most integrations start in. It is also the argument for deciding what to log deliberately rather than logging whatever the framework does by default.
What a good answer looks like
Not "a major provider, and they hold SOC 2". Something closer to:
Prompts go from our service to one gateway, which forwards to one of three named providers depending on the model requested. We log token counts, latency, model and a request id; we do not log prompt or completion content. The gateway retains those metadata records for N days. Each provider's retention and training terms are in their DPA, which we hold. Error tracking is configured to strip request bodies, and we verify that quarterly.
Every clause there is checkable, which is the point. The verifiability is what makes it a good answer, not the content of any individual claim.
Where to start if you have none of this
- Write down the hops. Just the list — most teams discover one they had forgotten while writing it.
- Grep for content in logs. Fix what you find before documenting anything.
- Collect the DPAs and sub-processor lists you already have the right to.
- Write the paragraph above for your own system, with the blanks filled.
That paragraph is the deliverable. It answers the security review, it tells your own team what the rules are, and writing it usually surfaces one thing worth fixing — which is the real reason to do it before someone asks.
The questions a buyer will put to you are covered in twelve data questions to ask an AI vendor; ours are answered on the security page, in the same shape.