Questions to ask AI vendors: procurement checklist for UK tech leaders

Every AI vendor evaluation should cover ten categories: data handling and residency; model provenance and performance; explainability and bias; security and adversarial resilience; compliance and contracts; implementation and SLAs; testing and acceptance on your data; change control and versioning; timeline and cost drivers; and exit terms and portability. Miss any one of them and you are signing a contract with gaps that will cost you later.

Three answers should stop a procurement process immediately. First, any vendor who refuses to sign an Article 28 Data Processing Agreement under UK GDPR is not a compliant processor — full stop. Second, a vendor who will not confirm a written, retroactive no-training opt-out is telling you that your confidential data may appear in future model outputs or be exposed to other customers after a model update. Third, a vendor who cannot support independent evaluation on a sample of your data is hiding something about performance.

Before you read further, here are the five questions to send today:

• Does your platform use our prompts, outputs or logs to train or fine-tune models, and will you confirm a retroactive opt-out in writing?
• Will you sign an Article 28 DPA and provide a named sub-processor list with change-notification commitments?
• Can you provide a model manifest covering architecture, named model versions in production, and training-data categories?
• What are your SOC 2 Type II or ISO 27001 scope and your most recent red-team or prompt injection test summary?
• What are your data export formats, deletion timelines, and model artefact handling at contract end?

Pro Tip: Send these five questions before a demo. A vendor who hedges or delays on any of them is showing you exactly how they will behave in a contract dispute.

Table of Contents

• One-page vendor questionnaire and scoring guide
• What to ask about data handling, residency and training controls
• Which models run, who owns them, and how do you verify performance?
• How to evaluate explainability, feature mapping and bias testing
• Security evidence and adversarial resilience: what to request
• Compliance, liability and contract clauses UK buyers must insist on
• Implementation, onboarding, support tiers and SLA expectations
• How to test vendor claims: pilot design and acceptance criteria
• Change control, versioning and preventing silent model updates
• Typical deployment timelines and primary cost drivers to budget for
• How to design a scored AI vendor questionnaire
• UK procurement must-haves and expert recommendations
• Key takeaways
• The question most procurement teams ask too late
• Gmdautomation: a managed AI deployment built for UK procurement requirements
• Primary references and authoritative sources for your RFP

One-page vendor questionnaire and scoring guide

A structured RFP that requires evidence and named contacts gets better responses and reduces the number of vendors that proceed to costly pilots. The table below maps each category to its core questions and a simple red/amber/green signal.

Any red signal in the mandatory columns (data handling, compliance, security) is a pass/fail failure. Convert amber answers into pilot requirements or contract negotiation items before proceeding.

• Mandatory pass/fail controls: signed DPA, written no-training opt-out, basic encryption at rest and in transit.
• Amber answers that survive to pilot: treat them as acceptance criteria the vendor must satisfy before production approval.
• Evasive answers: document them and escalate to legal or security review rather than accepting a verbal assurance.

What to ask about data handling, residency and training controls

This is the category where UK procurement has the least room to negotiate. Under UK GDPR, any vendor processing personal data on your behalf must be treated as a data processor, and a signed Article 28 DPA is a legal requirement, not a nice-to-have. Without one, sharing personal data with the vendor may be unlawful.

The no-training clause matters just as much. A written, retroactive term that prevents the vendor from using your confidential inputs, outputs, or logs to train or fine-tune models is the only protection against proprietary information appearing in another customer's model output after an update. Verbal assurances and policy pages are not sufficient — the clause must be in the contract and must apply retroactively to data already processed.

Ask these questions in writing:

• Are our prompts, files, outputs, telemetry and logs used to train, fine-tune or improve any model? Is the opt-out retroactive?
• Will you provide an Article 28 DPA with a named sub-processor list and a commitment to notify us before adding new sub-processors?
• Where is our data processed and stored? Which jurisdictions and data centres?
• What are the retention, deletion and backup timelines? How do you handle model artefacts derived from our data after contract termination?
• Can you provide proof of deletion (e.g. a signed certificate) within a defined period after contract end?

Contract clause template (minimum): "The Processor shall not use Controller data, including prompts, outputs, logs or derived artefacts, to train, fine-tune or improve any model. This restriction applies retroactively to all data processed under this Agreement. The Processor shall provide a signed deletion certificate within [30] days of contract termination, confirming deletion of all Controller data and derived model artefacts."

Pro Tip: During diligence, ask the vendor to walk you through their data flow diagram and point to exactly where your data touches model training pipelines. If they cannot show you the diagram, treat it as a red signal.

Which models run, who owns them, and how do you verify performance?

Generic statements about "secure infrastructure" hide model-level risks. A model manifest that breaks down architecture, training-data categories, and named model versions in production is the minimum you need for a meaningful risk assessment and migration planning.

Ask the vendor to confirm whether they use proprietary models, open-weight models (such as those from the Llama or Mistral families), or third-party API calls to foundation model providers. Each carries different risk: third-party API calls introduce an additional sub-processor and a dependency on that provider's own terms; open-weight models may carry licence restrictions; proprietary models may be harder to audit.

Performance on generic benchmarks such as MMLU or HumanEval rarely reflects how a model behaves on your actual workflows. Buyer data or representative synthetic tests give a far truer signal of operational fit. Require the vendor to run evaluation on your data or a controlled synthetic dataset before you commit.

Questions to include in your RFP:

• What is the model manifest? Name the models in production, their architecture type, and the broad categories of training data used.
• Do you wrap or fine-tune foundation models? If so, which base models and under what licence?
• What is your update cadence, and can we access a changelog and pin a model version during enterprise validation?
• Will you run an evaluation on our data or representative synthetic data and provide exportable results?

How to evaluate explainability, feature mapping and bias testing

Require a feature map before you sign anything. It should show which product features use AI, how AI affects output decisions or actions, and where a human remains in the loop. Without it, you cannot assess where GDPR data-subject rights obligations apply or where a model error could cause harm.

Model cards and decision logs are the standard explainability artefacts. Ask whether the vendor publishes model cards and whether decision logs are available at the individual-output level. For high-stakes use cases (credit, hiring, healthcare triage), explainability is both an ethical requirement and a legal one under UK GDPR's provisions on automated decision-making.

Bias testing claims are easy to make and hard to verify. Push past policy statements and ask for specific test results: which demographic groups were covered, what fairness metrics were used, and what mitigations were applied where disparate impact was found.

• Does the vendor publish model cards for models used in your deployment?
• Which product features use AI, and how does AI influence the output or action taken?
• What bias testing has been conducted? Which demographic groups? Which fairness metrics?
• How do decision logs support data-subject access requests under UK GDPR?
• What tools does the vendor provide to help your team explain individual outputs to end users or regulators?

Pro Tip: Turn vague fairness claims into a pilot criterion. Ask the vendor to run 200 synthetic cases that reflect your customer population's demographic distribution and provide the per-group accuracy breakdown. If they refuse, that is your answer.

Security evidence and adversarial resilience: what to request

SOC 2 Type II is a baseline, not a finish line. AI-specific adversarial risks — prompt injection, data extraction via crafted inputs, model inversion — require specialised red-teaming that a standard SOC 2 audit does not cover. Request the executive summary of the vendor's most recent penetration test and red-team report, and confirm it explicitly covers prompt injection and data extraction scenarios.

For enterprise AI security architecture, the access control layer matters as much as the certification. Check that the vendor supports SSO, MFA, role-based access control (RBAC), API key rotation, IP allow-listing, and customer-facing audit logs. If your security team cannot see who accessed what and when, you cannot investigate an incident.

Evidence checklist to attach to your security review:

• SOC 2 Type II report (executive summary, scope, and any exceptions noted)
• ISO 27001 certificate and scope statement (if applicable)
• Most recent penetration test executive summary (dated within 12 months)
• Red-team report covering prompt injection and data extraction tests
• Description of access controls: SSO, MFA, RBAC, API key management, audit log retention period
• Incident response plan with AI-specific event definitions and notification timelines
• Breach notification SLA confirming the 72-hour GDPR reporting window

Minimum acceptance criteria: current SOC 2 Type II or ISO 27001 in scope; red-team report dated within 12 months; documented breach notification SLA of 72 hours or less.

Compliance, liability and contract clauses UK buyers must insist on

The EU AI Act is now in force and applies to UK organisations that deploy AI systems affecting EU residents, even post-Brexit. Separately, UK GDPR imposes its own obligations. Know which risk category the vendor's system falls into under the EU AI Act, and ask whether a documented AI impact assessment is available for high-risk systems.

The IP question is frequently glossed over in vendor demos. Outputs generated by the AI may be subject to third-party copyright claims depending on training data provenance. Insist on an explicit indemnity covering third-party IP claims arising from model outputs, and confirm that your organisation owns the outputs it generates.

Legal checklist for in-house counsel:

• Signed Article 28 DPA with named sub-processors and change-notification commitment
• Written no-training clause, retroactive in scope
• IP ownership of outputs explicitly assigned to your organisation
• Indemnity for third-party IP claims arising from model outputs
• Documented liability caps and professional indemnity / cyber insurance levels
• Audit rights or third-party audit allowances for high-risk deployments
• Exit-period data export commitments and deletion certificate obligation
• Complementary User Entity Controls (CUECs) documented: which human-in-the-loop responsibilities belong to the vendor and which to your organisation

Defining CUECs before signature prevents vendors from offloading oversight responsibilities onto the buyer after go-live. Without them, "human in the loop" becomes your problem, not theirs.

Pro Tip: Any unresolved IP or liability item should become a pilot gating criterion. Tell the vendor: "We will not proceed to production until this clause is agreed." It is a faster negotiation lever than a legal letter.

Implementation, onboarding, support tiers and SLA expectations

Clarify what the base subscription covers before you compare prices. Implementation, runbooks, and ongoing optimisation should be included; custom integration work, fine-tuning, and dedicated support engineers are often separately priced. Get a written scope of services before the commercial proposal.

SLAs for AI deployments need to cover the model endpoint specifically, not just the web interface. An uptime guarantee that applies only to the dashboard is meaningless if the inference API is down. Ask for uptime commitments on the model endpoint, escalation response times for AI-specific incidents (such as a model producing harmful outputs), and credits tied to missed ML-specific targets.

Questions to ask:

• What does the base subscription include? What triggers additional charges?
• Does the uptime SLA cover the model inference endpoint specifically?
• What are the escalation paths and response times for AI-specific incidents?
• Who is the named implementation contact, and what is the handover process if they leave?
• Are runbooks for incident response and rollback included, or separately priced?

How to test vendor claims: pilot design and acceptance criteria

Benchmarks like MMLU or HumanEval rarely reflect enterprise failure modes. The only test that matters is one run on your data or a controlled synthetic dataset that mirrors your real workflows. Make a structured pilot a contractual prerequisite for production approval.

A well-designed pilot has three components: a defined test dataset (real or synthetic), a set of pass/fail thresholds agreed in advance, and exportable logs that allow your team or a third party to reproduce the results. Vendors who resist providing exportable logs or reproducible test scripts are preventing independent audit.

Testing plan template:

1. Define the test dataset. Use a representative sample of real data (anonymised if necessary) or synthetic data that mirrors your workflow's input distribution and edge cases.
2. Set acceptance thresholds. Agree metrics and pass/fail values before the pilot starts: accuracy, false positive rate, p95 latency, and failure mode frequency.
3. Require exportable artefacts. Logs, per-output traceability, and reproducible test scripts must be delivered as part of the pilot report.
4. Map results to contract milestones. Pilot success triggers transition to production and the first subscription payment. Failure triggers a defined remediation period or contract termination right.
5. Assign a named owner. One person on each side owns the pilot sign-off decision and is named in the contract.

Sample acceptance criteria list:

• Task accuracy ≥ agreed threshold on the test dataset
• False positive rate ≤ agreed threshold
• p95 latency ≤ agreed threshold at expected peak load
• Zero critical failure modes (e.g. harmful outputs, data leakage) during the pilot period
• Exportable logs delivered within five business days of pilot completion
• Reproducible test scripts verified by buyer IT team

Change control, versioning and preventing silent model updates

Silent or automated model updates can change production behaviour overnight. A model that passed your acceptance criteria in January may behave differently in March after a silent update. Procurement must require changelog access, written release notification lead times, and the contractual right to remain on a previous model version during enterprise validation.

API versioning is a related but distinct requirement. Breaking changes to an API without versioning cause production incidents that are expensive to diagnose and fix. Confirm that the vendor versions both their APIs and their model releases, and that you will receive written notice before a breaking change is deployed.

Contract clause language to insist on:

• "The Vendor shall provide written notice of any model update at least [14] calendar days before deployment to production environments. The Customer shall have the right to remain on the current model version for a period of [30] days following notice to conduct validation testing."
• "In the event that a model update causes a measurable regression in performance against the agreed acceptance criteria, the Vendor shall provide a rollback to the previous model version within [48] hours of a written request from the Customer."
• "The Vendor shall maintain a publicly accessible or customer-accessible changelog documenting all model and API changes, including the nature of the change, the date of deployment, and any known behavioural differences."

For API integration and versioning specifics, confirm that deprecated API versions remain available for a minimum notice period before sunset.

Typical deployment timelines and primary cost drivers to budget for

Most enterprise AI deployments follow a four-phase pattern: discovery and scoping, security and legal review, pilot, and production integration. The legal and security review phase is consistently the longest blocker — not the technology. Budget four to six weeks for it, not two.

Primary cost drivers to identify early:

• Per-call or per-token pricing: model at 10x and 100x your expected usage before signing.
• Fine-tuning and custom model training fees: often not included in base pricing.
• Data hosting and egress costs: particularly relevant for UK data residency requirements.
• Implementation professional services: scoped separately from the subscription in most contracts.
• Premium support tiers: named engineers and faster SLAs carry a material uplift.

Tips for accelerating procurement without increasing risk: run the security and legal review in parallel with the pilot scoping, not sequentially. Pre-agree the acceptance criteria before the pilot starts so there is no renegotiation at sign-off.

How to design a scored AI vendor questionnaire

A scored questionnaire converts subjective vendor impressions into a defensible shortlisting decision. The scoring model below uses a 0–100 scale with mandatory pass/fail gates that disqualify a vendor regardless of their total score.

Weighting structure:

1. Mandatory controls (pass/fail, not scored). Signed DPA, written no-training opt-out, basic encryption at rest and in transit. Failure on any of these eliminates the vendor from the process.
2. High-weight categories (15 points each). Data handling, security evidence, compliance and contracts. These carry the greatest legal and operational risk.
3. Medium-weight categories (10 points each). Model provenance, testing and acceptance, change control. Important but remediable through pilot conditions.
4. Advisory categories (5 points each). Explainability, implementation, timeline and costs. Useful for differentiation but not disqualifying.

Converting evasive answers:

• Vague claim with no evidence → require a specific artefact (red-team report, model card, DPA draft) as a condition of advancing to the next stage.
• Partial answer → convert to a pilot acceptance criterion that must be met before production approval.
• Refusal to answer → treat as a red signal and document it for the scoring record.

Sample questionnaire structure for an RFP or vendor portal:

• Section A (mandatory): DPA available? No-training opt-out in writing? Encryption at rest and in transit?
• Section B (scored, 15 points): Data handling questions 1–4, with evidence required for each.
• Section C (scored, 15 points): Security evidence questions 1–4, with artefacts requested.
• Section D (scored, 15 points): Compliance and contract questions 1–4.
• Section E (scored, 10 points each): Model provenance, testing, change control.
• Section F (scored, 5 points each): Explainability, implementation, costs.

A structured RFP that requires evidence and named owners accelerates selection and reduces the number of vendors that proceed to costly pilots. Set a minimum score threshold (for example, 70 out of 100) for shortlisting, and make the mandatory pass/fail gates non-negotiable.

UK procurement must-haves and expert recommendations

UK GDPR imposes direct obligations on data controllers that cannot be delegated to a vendor. The Article 28 DPA is not a formality — it must map processing locations, name sub-processors, and include a commitment to notify you before adding new ones. Processing locations matter for data residency: confirm whether data is processed in the UK, the EEA, or third countries, and whether appropriate transfer mechanisms are in place for any third-country transfers.

The EU AI Act introduces risk-category obligations for systems affecting EU residents. Even post-Brexit, UK organisations deploying AI in EU-facing workflows need to understand which risk category applies and whether a documented AI impact assessment is required. Ask the vendor directly, and request the assessment if the system falls into the high-risk category.

For AI governance in practice, the most overlooked protection is the CUEC definition. Without it, vendors routinely push human oversight responsibilities onto the buyer after go-live. Require the vendor to document which human-in-the-loop controls belong to them and which belong to your organisation before signature.

Legal checklist for counsel:

• Article 28 DPA with processing locations, sub-processor list, and change-notification commitment
• Written no-training clause, retroactive in scope, covering prompts, outputs, logs and derived artefacts
• IP ownership of outputs assigned to the buyer
• Indemnity for third-party IP claims from model outputs
• Liability caps and professional indemnity / cyber insurance levels confirmed in writing
• Audit rights or third-party audit allowances for high-risk deployments
• Exit-period data export commitments and deletion certificate obligation within a defined period
• CUECs documented and signed by both parties

Pro Tip: Use pilot gating as a negotiation lever. Any unresolved legal or security item becomes a condition that must be satisfied before the pilot transitions to production. It is faster and more enforceable than trying to renegotiate a signed contract.

Key takeaways

The single most important step in any AI vendor evaluation is to demand a signed Article 28 DPA, a written no-training opt-out, and a structured pilot on representative data before any production commitment.

The question most procurement teams ask too late

The standard procurement mistake is treating AI vendor selection like a SaaS purchase. You compare features, check the pricing page, run a demo, and sign. The problem is that AI systems are not static software. They update silently, they process your data in ways that may not be obvious from the interface, and their failure modes are probabilistic rather than deterministic. A bug in traditional software either breaks or it does not. A model regression degrades quietly, and you may not notice until a customer complains or a regulator asks questions.

The questions that matter most are the ones vendors find uncomfortable: show me the red-team report, give me the model manifest, let me run the pilot on my data and keep the logs. Vendors with nothing to hide answer these quickly. The ones who stall, redirect to marketing materials, or offer a "trust us" policy statement are telling you exactly what the contract will look like.

The other thing procurement teams consistently underestimate is the exit. Data portability and deletion commitments feel abstract when you are excited about a new capability. They feel very concrete when you are trying to leave a vendor who holds your data in a proprietary format and has no contractual obligation to delete model artefacts derived from your inputs. Get the exit terms in writing at the start, not as an afterthought.

The scoring model in this article is designed to make those uncomfortable questions systematic. When a vendor knows their answers will be scored against evidence requirements and that evasive answers become pilot gating criteria, the quality of their responses improves considerably.

Gmdautomation: a managed AI deployment built for UK procurement requirements

If the checklist above describes what you need from a vendor, Gmdautomation is built to answer it. UK businesses working with Gmdautomation get enterprise-grade AI automation delivered as a fully managed service: implementation, compliance controls, ongoing optimisation, and support are all included in a predictable monthly subscription with no upfront capital cost.

That means the procurement questions in this article are ones Gmdautomation is designed to answer directly. Compliance documentation, onboarding plans, and security evidence are part of the deployment, not extras to negotiate. For procurement teams who want to move from checklist to live deployment without managing a complex multi-vendor integration, the managed model removes most of the implementation risk.

To see how Gmdautomation maps to your procurement requirements, request a managed AI deployment and ask for a compliance pack alongside the pilot proposal.

Primary references and authoritative sources for your RFP

Attach these to your vendor questionnaire or share with legal and security reviewers. Keep responses and evidence in a central diligence repository and score them consistently against the criteria in this article.

• AI Procurement Checklist: 47 Questions Before Buying AI Tools (ThinkTech) — covers seven categories including data handling, model transparency, compliance, and exit strategy. Useful as a secondary checklist to cross-reference against your scored questionnaire.
• AI Vendor Due Diligence: 30 Questions in Five Categories (ThinkTech) — covers the model manifest, no-training clause, and representative testing requirements. Attach to the model provenance and testing sections of your RFP.
• Free AI Vendor Assessment Checklist: 25 Questions (VerityAI) — strong on UK GDPR Article 28 obligations and SOC 2 / red-team evidence requirements. Share with your legal and security reviewers.
• Enterprise AI Vendor RFP: 40 Questions (SitePilot) — covers change management, CUEC definition, and exit terms. Useful template for structuring the RFP itself.
• Enterprise AI Vendor RFP Questions (Worqlo) — practical guidance on structured RFP design and converting evasive answers into evidence requirements.

Documents to request from vendors and store in your diligence repository:

• Signed Article 28 DPA with sub-processor list
• SOC 2 Type II executive summary (current year)
• ISO 27001 certificate and scope (if applicable)
• Red-team and penetration test executive summary (dated within 12 months, covering prompt injection)
• Model manifest with named model versions and training-data categories
• Feature map showing which product functions use AI
• Bias test results with demographic coverage and fairness metrics
• Changelog and release notification policy
• Exit terms: data export formats, deletion timelines, model artefact handling

Recommended

• AI in procurement operations: a 2026 executive guide
• Types of AI middleware solutions: a UK guide
• Managed AI service vs in-house: 2026 guide for UK businesses
• Why system integrators adopt AI platforms: 2026 guide