The Vendor Security Questionnaire That Actually Catches AI Risk

The Vendor Security Questionnaire That Actually Catches AI Risk

If you have sent out a SIG questionnaire in the last twelve months and felt confident it covered your AI vendor exposure, I have some bad news. The Shared Assessments SIG (Standardized Information Gathering) questionnaire and the Cloud Security Alliance's CAIQ (Consensus Assessments Initiative Questionnaire) are both excellent instruments for traditional IT and cloud risk. They were not built to surface the specific ways an AI system can compromise your data, your compliance posture, or your customers. Both organizations know this, and both are updating. But the gap between where those tools are today and where your actual AI risk lives is worth examining closely.

Where SIG and CAIQ Stand Right Now

Shared Assessments released SIG 2024 with expanded language around "emerging technology," which includes AI and machine learning processing. The updates touch on data governance, model transparency, and automated decision-making. These additions live mostly within the existing domains for Information Security (Domain D), Privacy (Domain H), and Operational Resilience (Domain K). They are useful starting points. But they tend to treat AI as a subcategory of data processing rather than a fundamentally different risk vector.

The CSA updated CAIQ v4 to align with the Cloud Controls Matrix (CCM) v4, which introduced controls around AI transparency and accountability under the "Artificial Intelligence" domain (AIS). This was a meaningful step. The AIS domain includes questions about training data provenance, model explainability, and bias monitoring. CSA also published its AI Safety Initiative guidance in late 2023, which provides supplemental context.

Both frameworks are moving in the right direction. The problem is pace. The EU AI Act entered into force on August 1, 2024, with its first set of obligations (the prohibitions in Title II) applying from February 2, 2025. NIST released AI RMF 1.0 in January 2023 and the companion AI RMF Playbook shortly after. Colorado's SB 24-205, the first comprehensive U.S. state law regulating AI in high-risk decisions, was signed in May 2024. Regulatory specificity is outrunning the questionnaire update cycle.

The Questions SIG and CAIQ Do Not Ask

Here is where it gets practical. Below are the categories of questions that a standard SIG or CAIQ deployment will miss or underweight, and that you should be supplementing into your TPRM process now.

Training Data Rights and Contamination

Standard questionnaires ask whether a vendor protects data at rest and in transit. They rarely ask whether your data, once ingested by an AI system, becomes part of a training corpus. This matters enormously. If your vendor fine-tunes models on customer inputs, your regulated data may influence outputs served to other clients. You need to ask:

• Does any customer data, including prompts, queries, or uploaded documents, enter a training or fine-tuning pipeline?
• If so, what is the retention period for training data, and can specific records be deleted from a trained model (practically, not theoretically)?
• Has the vendor conducted an intellectual property review of its foundational training data to assess copyright or licensing risk under pending litigation like The New York Times Co. v. Microsoft Corp. (N.D. Cal., filed Dec. 2023)?

Model Provenance and Supply Chain

Many vendors building AI products are not training their own models. They are wrapping API calls to OpenAI, Anthropic, Cohere, or open-source models from Hugging Face. Your vendor's security posture is only as strong as the model provider underneath, and that subprocessor relationship often goes unexamined. Ask:

• What foundation model(s) does the product rely on, and who are the upstream model providers?
• Does the vendor have contractual flow-down provisions with model providers covering data handling, retention, and breach notification?
• If using open-source models, what is the vendor's process for evaluating model integrity, including checks against known model poisoning or backdoor risks?

Output Reliability and Hallucination Controls

This is the one that keeps general counsel up at night, and it has no real analog in traditional vendor risk. A database either returns the correct record or throws an error. An LLM can fabricate a plausible but entirely fictional answer with high confidence. If your vendor's AI product generates outputs that inform clinical decisions, financial advice, legal research, or student assessments, you need specifics:

• What mechanisms does the vendor use to detect and mitigate hallucinated outputs (retrieval-augmented generation, confidence scoring, human-in-the-loop review)?
• What is the vendor's measured hallucination rate on tasks comparable to your use case, and how was that rate benchmarked?
• Does the vendor carry professional liability or errors-and-omissions insurance that explicitly covers AI-generated output failures?

Regulatory Alignment and Incident Response

Standard breach notification questions assume a traditional data breach: unauthorized access, exfiltration, ransomware. AI systems introduce novel incident types. A model inversion attack could allow someone to reconstruct training data. A prompt injection could cause the system to bypass its safety controls and expose sensitive information. Your questionnaire should probe:

• Does the vendor's incident response plan specifically address AI-related incidents, including prompt injection, model extraction, and adversarial manipulation?
• For vendors operating in the EU, has the vendor completed a conformity assessment or gap analysis against the EU AI Act's requirements for the risk category applicable to your use case?
• Does the vendor maintain an AI-specific risk register separate from its general information security risk register?

Automated Decision-Making and Explainability

If the vendor's AI is making or materially influencing decisions about individuals, you are likely on the hook under GDPR Article 22, Colorado's SB 24-205, or sector-specific rules like the CFPB's guidance on adverse action notices in credit decisioning (CFPB Circular 2022-03). The vendor needs to help you comply:

• Can the vendor produce a plain-language explanation of how a specific output or decision was reached for a specific individual?
• Does the vendor support your obligation to provide opt-out or human review mechanisms for automated decisions?
• What bias testing has been conducted, on what populations, and at what frequency?

Building the Supplemental Questionnaire

The practical move here is not to abandon SIG or CAIQ. They remain valuable for baseline IT and cloud security assessment, and your assessors already know how to use them. Instead, build a supplemental AI risk module that layers on top. Keep it to 25 to 40 questions. Map each question to a specific regulatory requirement or risk scenario so that respondents understand why you are asking. Weight the questions by the criticality of the AI function to your operations. A vendor using AI for internal log analysis is a different risk profile than one using AI to generate customer-facing content in a regulated context.

Version your supplement and plan to update it at least twice a year. The regulatory landscape here is moving fast enough that an annual review cycle will leave you exposed. The OCC's spring 2024 guidance on third-party risk management for banks (OCC Bulletin 2024-16) already flags AI as a specific area requiring enhanced due diligence beyond standard TPRM frameworks.

How FirmAdapt Addresses This

FirmAdapt was built with the assumption that regulated companies need to answer these questions about their own AI deployments, not just ask them of vendors. The platform's architecture keeps customer data strictly isolated from any training pipeline, maintains full audit trails of how outputs are generated, and maps its own controls to frameworks including NIST AI RMF, the EU AI Act, and sector-specific requirements like HIPAA and GLBA. When your TPRM team sends you a questionnaire with the kinds of questions described above, FirmAdapt gives you documented, verifiable answers rather than vague assurances.

For organizations running their own TPRM programs, FirmAdapt's compliance mapping can also help structure and maintain that supplemental AI risk questionnaire, tying each question to the evolving regulatory requirements that justify it. The goal is to make AI vendor risk assessment as rigorous and repeatable as the traditional security assessments you already run.