Enterprise RAG Architecture for Sensitive Financial Data
The short answer
A compliant retrieval-augmented generation (RAG) system in financial services enforces the same access rules as the source systems at retrieval time, not after the answer is generated. That means permission-aware indexing, document-level entitlements passed with every query, data loss prevention on inputs and outputs, and an audit trail linking every answer to the documents it used.
Why RAG is harder in a bank than in a demo
A demo RAG system indexes a folder and answers questions. In a bank, that folder contains material non-public information, client data, and research with information barriers. If the vector index ignores those boundaries, the assistant becomes the easiest way to leak them.
Reference architecture: the control points
| Layer | Control | Why it matters |
|---|---|---|
| Ingestion | Classify documents and carry source ACLs as metadata | Entitlements travel with every chunk |
| Index | Separate or filtered indexes for information barriers | Prevents cross-wall retrieval |
| Query | Pass the user's identity and roles into retrieval filters | Users only retrieve what they could already open |
| Generation | Approved model endpoint with no training on bank data | Keeps data inside agreed terms |
| Output | DLP scanning, citation of sources, refusal on low confidence | Reduces leakage and unsupported answers |
| Audit | Log prompt, retrieved documents, answer, and user | Supports supervision and investigations |
Design checklist
- Inventory source systems and their permission models before building the index.
- Sync entitlement changes (joiners, movers, leavers) into the index on a defined schedule.
- Filter at retrieval, never by post-processing an answer that already saw restricted text.
- Require citations so reviewers can verify each answer against source documents.
- Apply DLP to prompts and outputs for account numbers, PII, and restricted terms.
- Retain logs per the firm's record-keeping policy and make them searchable for compliance.
- Red-team for prompt injection embedded in ingested documents.
Start narrow
The fastest safe path is one well-governed corpus — policies and procedures, or product documentation — with a single permission model. Prove the control points there before adding client or deal data.
Frequently asked questions
What is enterprise RAG?
Retrieval-augmented generation retrieves relevant internal documents and gives them to a language model as context for an answer. Enterprise RAG adds identity, permissions, logging, and data controls so it can run on sensitive internal data.
How do you enforce access control in a RAG system?
Store each document's access rules as metadata at ingestion and filter retrieval by the requesting user's identity and roles, so the model never sees content the user could not already access.
Does RAG stop hallucinations?
It reduces them by grounding answers in retrieved documents, but does not eliminate them. Citations, confidence thresholds, and human review for material uses remain necessary.
Can RAG respect information barriers?
Yes, if barriers are modelled explicitly — through separate indexes or mandatory filters — and tested. It should be validated with compliance before any cross-business deployment.
Official sources
- FTC: Gramm-Leach-Bliley Act and Safeguards Rule
- NIST SP 800-53 Rev. 5: Security and Privacy Controls
- NIST AI 600-1: Generative AI Profile