Model Risk Management for GenAI in Banking: Applying SR 11-7 to LLMs
The short answer
SR 11-7 and OCC Bulletin 2011-12 still apply when a bank uses a large language model in a decision or reporting process. Its three pillars — sound development and use, independent validation, and governance — remain the frame. What changes is how you evidence them: LLM outputs are non-deterministic and vendor models are opaque, so validation shifts toward use-case scoping, output testing, ongoing monitoring, and documented human oversight.
Why GenAI strains traditional model validation
Classic model risk programs were built for predictive models with stable inputs, transparent features, and repeatable outputs. A foundation model accessed through an API breaks several of those assumptions: the bank cannot see training data, the vendor may update weights, and the same prompt can produce different answers.
The guidance itself is principles-based, which is why it can stretch. SR 11-7 defines a model broadly and expects rigor commensurate with risk. The practical question is not whether it applies, but how to make each pillar testable for an LLM use case.
Mapping SR 11-7 pillars to LLM use cases
| SR 11-7 pillar | Traditional evidence | GenAI adaptation |
|---|---|---|
| Development and use | Model design, data, assumptions | Defined use case, prompt and retrieval design, approved data sources, intended users |
| Conceptual soundness | Theory and feature review | Vendor documentation, known limitations, fitness for the specific task |
| Outcomes analysis | Back-testing against actuals | Curated evaluation sets, accuracy and hallucination testing, bias checks |
| Ongoing monitoring | Performance thresholds | Output sampling, drift on vendor updates, user feedback, incident logging |
| Governance | Inventory, policy, roles | Inventory entry per use case, risk tier, owner, human-review requirements |
| Third-party risk | Vendor due diligence | Model version control, change notifications, data-handling terms |
A practical MRM checklist for a GenAI use case
- Register each GenAI use case (not just the vendor model) in the model inventory with a named owner.
- Assign a risk tier based on decision impact, customer exposure, and regulatory reporting use.
- Document intended use, out-of-scope uses, and the human-review step for material outputs.
- Build an evaluation set from real, permissioned examples and record pass thresholds before go-live.
- Test for hallucination, inconsistency, prompt injection, and disparate outcomes where relevant.
- Agree vendor change notification and re-test whenever the underlying model version changes.
- Monitor sampled outputs in production and log incidents into the existing MRM issue process.
Where banks usually get stuck
The most common blocker we see is classification: teams debate whether an assistant is a 'model' and nothing moves. A faster path is to treat the use case as in scope, tier it, and let low-risk internal uses run with lighter validation while customer-facing or decision-support uses get full review.
Frequently asked questions
Does SR 11-7 apply to generative AI and LLMs?
SR 11-7 defines models broadly and is principles-based, so many banks treat LLM use cases that inform decisions or reporting as in scope. Whether a specific use is covered should be decided by the bank's model risk function and documented.
How do you validate a non-deterministic LLM?
Validation shifts from reproducing a single output to measuring behaviour across a curated evaluation set: accuracy, consistency, hallucination rate, and failure modes, with thresholds agreed before deployment and re-tested after model changes.
What about vendor models the bank cannot inspect?
SR 11-7 expects vendor models to be incorporated into the bank's risk management. In practice that means vendor documentation, contractual change notification, version pinning where possible, and stronger outcomes testing to compensate for limited transparency.
Does every GenAI use case need full validation?
No. The guidance expects rigor commensurate with risk. Low-impact internal productivity uses can carry lighter controls, while customer-facing or decision-support uses warrant full independent review.
Official sources
- Federal Reserve SR 11-7: Guidance on Model Risk Management
- OCC Bulletin 2011-12: Sound Practices for Model Risk Management
- NIST AI 600-1: Generative AI Profile
This article is general information, not legal or regulatory advice. Confirm how these rules apply to your organisation with your compliance team or counsel.