When your application sends a request to an LLM API, your customer data — the message content, potentially including PII or confidential business context — is processed by a third party. That third party is your AI provider. Standard vendor security questionnaires ask about SOC 2, encryption at rest, and access controls. For AI providers, these are necessary but not sufficient.
Questions Standard Questionnaires Miss
Is our data used for training?
Most enterprise AI providers offer opt-out from training data use, but it is often not the default. If your API requests are used to improve the model and those requests contain customer data, that data is effectively being shared with all future users of the model. Confirm the data retention and training policy in writing.
What is the data residency for inference?
API requests may be routed to data centers in any region unless you specify otherwise. For GDPR compliance, data processed by an EU customer cannot be routed to the US without adequate transfer mechanisms. Confirm where inference happens for each API endpoint you use.
What is the prompt injection incident response process?
If your users successfully inject instructions that cause the model to exfiltrate data, who is responsible and what is the response timeline? This is a novel incident category that most providers don't have explicit runbooks for. Ask what they will do and who will do it.
What is the BAA/DPA status for regulated data?
Processing health data through an LLM API requires a signed Business Associate Agreement from the provider. Processing EU personal data requires a Data Processing Agreement. Confirm both are available and review their terms — not all BAAs cover AI inference workloads.
Anthropic, OpenAI, and Google all publish their enterprise data handling policies publicly. Review the current version before each contract renewal — the policies have changed significantly in 2025-2026 as enterprise adoption has grown.