What a model card should tell a bank
We reviewed the documentation for twelve models in use at financial institutions in the region. Most would not pass a basic procurement check.
Dr. Fatima Noor
Research Lead · Safety, Alignment & Governance
Over the past year the Safety, Alignment & Governance Lab reviewed documentation for twelve models in production at banks and payment companies — with their cooperation, under NDA. Our conclusions are general enough to share.
Three gaps
No evaluation on local data. Nine of twelve cards reported benchmark results from the vendor with no evaluation on the institution’s own transactions or customers.
No breakdown. Only two cards reported performance by group — region, gender, account age. Aggregate accuracy hides the failures that regulators will ask about.
No limitations. Five cards had no limitations section at all.
What to require
We publish a procurement checklist in our public-sector evaluation guide. The short version: evaluation on your data, results by group, calibration, a limitations section, and the right to re-run the evaluation after every model update.
- governance
- finance
- evaluation
Get the next note by email
One considered letter a month on research, admissions and what we are building.
More in Perspectives
Why every course at the institute ends with an evaluation, not a demo
A note from the Dean on the pedagogy behind the academy — and why we grade validation plans more heavily than model accuracy.