The AI Questions in a Bank's Vendor-Risk Questionnaire — and What Evidence Answers Them
Banks now ask AI vendors to prove what their models did, not to describe it. The question clusters in a third-party risk review, and what closes each.
Who this guide is for
AI vendors selling into banks, fintechs and insurers
TL;DR — The questionnaire has stopped asking what your AI is designed to do and started asking what it actually did. A policy document answers the first question. Only a per-run record answers the second, and that is where most AI vendors stall.
If you sell software with AI in it to a bank, an insurer or a regulated fintech, you have met the third-party risk questionnaire. It used to be a security document: encryption, access control, subprocessors, breach notification, a SOC 2 report. You answered it once, kept the responses in a spreadsheet, and reused them.
That spreadsheet no longer clears the review. Since DORA came into force in January 2025, a financial entity's ICT third-party risk obligations flow down to its suppliers — and the AI you ship is ICT. The reviewer on the other side is not asking out of curiosity. They have their own obligation to evidence, and they can only satisfy it with something you give them.
What actually changed in the questions
The shift is narrow but it breaks most answer libraries. The old questions were about design: what does the system do, who can access it, what happens if it fails. The new questions are about the record: show us that it did what you say, for a specific decision, on a specific date.
| Design-era question | Evidence-era question |
|---|---|
| Do you log AI activity? | Show us the log for a decision made on 14 March, including the inputs and the model version. |
| Is there human oversight? | Show us a case where a human overrode the model, and who they were. |
| Do you have a model inventory? | Which model version served this customer's traffic last quarter, and when did it change? |
| Do you monitor for drift? | What did you detect, when, and what did you do about it? |
| Is data handled appropriately? | Where did this prompt's content physically go, and who could read it? |
A vendor can answer every question in the left column with a well-written policy. The right column cannot be answered with a document at all — it needs a record that was captured at the time, that nobody can retroactively edit, and that maps to the control the reviewer is testing.
The seven clusters, and what closes each one
1. Model inventory and versioning
What they ask: which models are in scope, who supplies them, what version served production, and how changes are controlled. What closes it: a per-run record carrying the model identifier and version, so the inventory is derived from what actually ran rather than maintained by hand. A hand-maintained inventory is accepted until the reviewer finds one run that contradicts it.
2. Record-keeping and traceability
What they ask: can you reconstruct a specific automated decision. What closes it: a durable record per run — inputs, outputs, the model, the timestamp, the identity that triggered it — retained for a defined period. This is the cluster that maps most directly to EU AI Act Article 12, and it is the one most vendors discover they cannot answer, because application logs were built for debugging and get rotated in days.
3. Human oversight
What they ask: where a human can intervene, and evidence that the mechanism is real. What closes it: records of approvals, overrides and escalations — with the identity of the person and the outcome. A screenshot of an approval button is a design answer. A list of the times somebody actually pressed it is an evidence answer.
This cluster fails quietly. Teams describe an oversight mechanism that exists in the product and is switched off in production, or that fires so rarely nobody noticed it stopped firing. The reviewer asks for instances, not for the feature.
4. Data handling and residency
What they ask: what content leaves the bank's boundary, where it lands, who can read it, and which subprocessors touch it. What closes it: a precise statement of what is held versus what is processed, backed by configuration the reviewer can verify. Be exact here. Vendors lose credibility on this cluster by over-promising — 'your data never leaves your environment' is rarely true in full, and a reviewer who finds one counter-example re-opens every other answer you gave.
5. Operational resilience
What they ask: your incident history, detection and response times, dependency on upstream model providers, and what happens when a provider degrades. What closes it: incident records and a dependency map that names your model providers as what they are — fourth parties in the bank's own supply chain. Under DORA this is not a courtesy disclosure; the financial entity has to register it.
6. Access control and identity
What they ask: who and what can invoke the model, how non-human identities are governed, and how access is reviewed. What closes it: attribution on every run — which agent, which service identity, which end user — and evidence of periodic review. The novel part is non-human identity: an autonomous agent acting with delegated permissions is an access-control subject, and most access reviews still only enumerate people.
7. Change management
What they ask: how a prompt, model or policy change reaches production, who approved it, and how you would roll it back. What closes it: a versioned trail of the change with the approver, plus the ability to point at which version was live during any given window. Prompt changes are the gap here — they change behaviour as much as a model swap and frequently ship outside the release process.
The three answers that stall a deal
- "We can produce that on request." The reviewer hears: it does not exist yet. If the record is not being captured today, it cannot be produced for last quarter — and last quarter is usually what they ask for.
- "Our platform is SOC 2 certified." SOC 2 covers your organisation's controls. It says nothing about what a specific model did on a specific decision, which is what an AI-specific question is asking.
- "That is handled by our LLM provider." Under DORA the financial entity has to understand its own supply chain. Pointing upstream adds a party to their register; it does not remove the question from you.
Why the deadline is closer than the AI Act suggests
It is worth being precise about timing, because a lot of published guidance is out of date. The EU AI Act's high-risk obligations under Annex III — which cover creditworthiness assessment and risk pricing in life and health insurance — were deferred by the Digital Omnibus of May 2026 and now apply from December 2027, not August 2026. If you read an article dated 2026 that still says August 2026, it predates that change.
The deferral moved the statutory deadline. It did not move the commercial one. DORA has been in force since January 2025, supervisors are asking now, and the vendor-risk questionnaire on your desk this quarter is not waiting for 2027.
That is the practical shape of it: the regulation that will eventually carry the penalty is on the horizon, and the review that decides whether you close this deal is on your desk. Vendors who prepare for the second are ready for the first.
Preparing before the questionnaire arrives
Pick one real decision your AI made last month. Try to reconstruct it end to end — inputs, model version, output, who could have intervened. Whatever you cannot reconstruct is your gap list, and it is more honest than any self-assessment.
Check your retention. Debug logs measured in days will not answer a question about last quarter. The record you need is a different artefact with a different lifetime.
Separate what you can prove from what you can only assert, and write both lists down. A reviewer will find the difference; it goes far better when you found it first.
Assemble the answers into one artefact you can hand over, rather than a thread of email attachments. Reviews are repeated — the second bank asks most of what the first one asked.
Where Dobby fits
Dobby connects to your AI activity out-of-band — it reads run telemetry from CrewAI, LangChain, OpenAI, Google ADK or a custom SDK, and never sits in the request path. It scans each run against the frameworks that apply (EU AI Act, DORA, SOC 2) and exports an audit-ready evidence pack: an executive summary, a control matrix, a severity-ranked gap report with remediation, the underlying findings, framework coverage, and a SHA-256 manifest so the recipient can verify nothing changed after export.
The verdict per control has four states rather than two: compliant, violated, needs-review, and unverifiable. That last state matters for exactly the reason this guide is about. When the telemetry is not sufficient to prove a control, Dobby says so instead of passing it. A pack that quietly reports compliance it cannot demonstrate is worse than no pack, because the reviewer is the one who finds out.
Non-custodial by design: for the Fintech AI Evidence module, raw prompt and response content is routed to a dataset you own rather than held by Dobby. Run metadata and verdicts are processed in Dobby's managed control plane.