General purpose AI tools like ChatGPT often require extensive training and fine-tuning to create reliably high-quality output for specialist and domain-specific tasks.
And public models’ scopes are simultaneously limited to and diluted by training materials. Output is based on their range of unidentified training material so it may diverge from output that needs to be anchored to defined sources of truth.
In financial services organisations, information flow and turnover is constant. Teams create, receive, and send hundreds of pages worth of documentation, reports, and other intel daily.
If an AI tool is unable to continuously ingest that flow of new information and data, which then informs its output, the tool provides limited value.
But retraining an LLM on the new documentation is impractical and expensive; the effort and investment required often reverses ROI gains.
A better alternative is adding a retrieval augmented generation (RAG) layer to AI systems.
RAG catalogues and retrieves up-to-date documentation like market analyses, reports, regulatory updates and changes, etc., for LLMs and other GenAI tools. With access to contextually accurate information, the tool’s output is precise and comes without hallucinations.
The use of RAG models in banking and fintech gives AI tools the flexibility to access new information. In the long-term as organisations grow, RAG makes secure AI document retrieval for finance, as well as other sectors working with highly sensitive data,
possible.
As leadership and technical teams at financial services organisations think strategically about how to build innovative systems to leverage AI, high-ROI, RAG-driven systems will be a top priority.
ROI, as you know, is achieved either by saving resources, or generating new income.
The top 3 out of 5 high-ROI uses of RAG models in banking and fintech do both: generate new income and save. Use cases 4 and 5 solely save.
1. Search and internal knowledge bases.
2. Investment research and market intelligence.
3. KYC and compliance document processing
4. Risk and fraud detection systems
5. Intelligent customer service
1. Enterprise search and internal knowledge bases: RAG in knowledge retrieval for employee tools
At mature start-ups, mid-level, and enterprise financial services, AI document retrieval run by RAG removes bottlenecks. It gives users fast access to accurate insights tucked away in complex knowledge base that would otherwise require considerable effort
to track down, extract, and consolidate.
A RAG-based system can give an employee role-based, secure access to every single relevant document an organisation has accumulated in the span of its existence.
There are a handful of enterprise-level organisations already using RAG-driven LLM systems to build AI assistants for internal teams, including Morgan Stanley and Spain’s BBVA.
Morgan Stanley built a GPT-4-based assistant for their over 16,000 advisors. With it, advisors instantly search over 100,000 internal documents to automatically generate client-meeting debriefs, reports, and action items.
BBVA added a RAG layer to their internal GenAI tool. It serves as a search resource and knowledge base for simpler data analysis and employee Q&A.
The implications are impressive. Rather than manually checking documents to find a specific statistic from 2020, for example, an advisor can type the query into the AI assistant’s chatbox. RAG forces the assistant to provide a cited answer leading directly
to the original source document.
Immediate access to the sum of an organisation’s documentation and gathered insights can improve efficiencies at every level and team.
That includes software development teams, whose work relies on consistent architectural and design principles. A RAG-augmented LLM, for example, would let them compare code and design ideas to current guidelines and past decisions and documents previous
teams made.
The impact on both code consistency and accuracy is monumental, especially for established banks catching up to smaller, digitally-native competitors. If making software development more efficient is relevant to your organisation, you can read about specific
AI models and tools that supplement developers’ work here.
With RAG, as long as the approved documents are organised in databases and available to the model, they are used to build context for each output.
That includes unstructured data that doesn’t fit a predefined format, like PDFs, visual resources like graphs and tables, etc. RAG models are able to structure those documents for easy retrieval.
And RAG models pull from a variety of relevant sources to generate contextually rich responses, making them especially useful in systems with a depth of historical data in various formats.
ROI
The easy-to-spot ROI in this case comes primarily from efficiency and speed. If employees can immediately access contextually accurate reservoirs of historical company data, their work is more precise and efficient.
But where this use case gets interesting is its impact on time-to-market and competitiveness. With reduced timelines and spend, and improved accuracy it’s possible to release better products and services faster, and require fewer resources to do so.
In turn, it’s easier to differentiate your organisation, and pivot if necessary. And to drive revenue in an increasingly competitive market.
One example is Goldman Sachs’s RAG-augmented AI assistant that retrieves internal knowledge for staff, launched after one year of experimenting, testing, and fine-tuning. In the first year they reported 20%-29% increased overall productivity. But their software
developers in particular are 20% more productive using it in tandem with other AI-driven tools like GitHub Copilot.
An organisation that rolls out a new feature in 10 months instead of 12 has a better chance at capturing market share and quickly responding to customer needs, yielding commercial benefits beyond cost savings.
But even the initial cost savings implications of a 20% increase in software developer productivity alone are compelling.
2. Investment research and market intelligence: Using LLMs + RAG for up-to-date financial insights
Another advantage of RAG in banking and fintech is its ability to access and accurately summarise an immense amount of information quickly.
The sheer amount of documentation required in financial services makes RAG compelling. Think of investment banking or wealth advisory, which require a variety of resources like market research, earnings call transcripts, economic data, etc.
RAG models quickly sift through this text-heavy data, fetch relevant information, and provide concise outputs. An LLM+RAG setup is like having an incredibly well-informed and fast research assistant available and at each of your employees’ desks 24/7.
Systems using a RAG layer can
- Accurately summarise lengthy equity research reports into key bullet points.
- Quickly extract notable quotes from earnings calls.
- Compare viewpoints across multiple analyst reports, etc.
You ask, “what’s the consensus on inflation in the latest Federal Reserve communications?” It pulls relevant excerpts from numerous Fed speeches and minutes to give a consolidated answer.
A real-world system utilising RAG is J.P. Morgan’s private tool, IndexGPT. It’s a portfolio management tool that scans news and identifies companies related to a given investment theme, including less obvious groups indirectly benefiting from a trend.
IndexGPT creates thematic stock indexes based on the organisations related to that theme. As a result, advisors scopes are automatically widened to include those that may have been overlooked or misjudged.
ROI
In J.P. Morgan‘s case, diversified thematic stock indexes can be produced almost immediately: a process that once required weeks to months now takes hours.
One byproduct of this efficiency from RAG-driven LLMs in investment banking is time and lighter mental load. Analysts and advisors are freed from hours of manual data gathering and can instead cover more ground and focus on higher-value judgment calls or
urgent tasks.
Less pressure and more time to pursue and strategise the approach to big-ticket items means smarter decisions.
But a less obvious gain is the generation of new income by improving depth of research. For example, what J.P. Morgan has done with IndexGPT, using it to diversify their thematic stock index to attract more diverse investors.
And by partially automating the search for emergent trends, investors can identify and capitalise on them more quickly, beating competitors.
3. KYC, AML and compliance document processing: How RAG assists in document summarisation and matching
This is an AI use case that is likely familiar.
KYC files and compliance checks require tedious attention to detail to cross-reference complex documents, making them a shoo-in for automation.
Manual processing of complex documents also increases errors, resulting in financial penalties. Last year, financial organisations in the US paid over $3 billion in AML penalties alone.
RAG models give AI-based systems, like LLMs and AI assistants, have secure one-time access to customer documentation necessary for KYC, AML, and compliance processing, including IDs, financial statements, credit reports, etc.
RAG forces the system to source outputs from documentation, freeing users from hallucinations and reducing the need for additional verification.
RAG also provides a secure way for compliance officers to quickly draft narratives for SARs (Suspicious Activity Reports) that are automatically consistent with internal and external requirements.
With each output, RAG leaves a clear trail of documents to simplify verification. Crucially, this transparency makes the system understandable to regulators and improves the overall system’s explainability.
ROI
ROI comes from both cost savings and faster throughput, as well as better compliance.
The more straightforward cost savings are extracted from a dramatic decrease in time and effort spent on onboarding and general processing. Document-heavy tasks that fall to expensive financial analysts and lawyers for first-pass processing can be semi-automated.
Automating, or even semi-automating, these tasks has a significant impact on overall costs.
One good example is J.P. Morgan. In 2022, they processed 155,000 KYC files with a workforce of 3,000.
After integrating AI tools, including IndexGPT, an internal LLMs, and chatbots, they reported a productivity increase of 90%, processing 230,000 files with 20% fewer employees.
While the exact impact that gained efficiency had on their ROI isn’t publicly available, a 90% productivity increase indicates it was substantial.
The more obscure but potentially more impactful leg up is improved accuracy in data entry and oversight, and faster turnaround. A system that sources inputs from internal documents directly reduces the opportunity for human error.
Fewer omissions and errors result in fewer compliance fines and remediation costs. And faster turnaround means customers can begin using products and services faster, generating new income.
4. Risk and fraud detection: Anomaly detection, real-time document retrieval and context
Precision and pattern detection make fraud detection and risk analysis using RAG models another high ROI use case.
RAG models pull data from multiple sources in real time, like customer documents or account history, for comprehensive analysis of transactions to identify fraud. These systems can use an organisation’s historical fraud intelligence alongside GenAI, like
LLMs, to perform a context-aware scan for abnormalities.
A well-built system compares current activity to past fraud incidents, mapping variables with identity clustering to evaluate ongoing transactions and accounts.
As soon as an anomaly is detected, the system flags it, creates a report, and sends it for human review. Though I imagine soon part of the initial review process will be automated as well, especially in lower-risk, data-rich contexts where RAG systems beat
human speed and accuracy.
One real-world example is BBVA’s AI agents. Their team has begun experimenting with agents to support technical assessment analysts on the cybersecurity team.
Their RAG-augmented agent analyses phishing messages in seconds, taking much of the burden off the analysts tasked with assessing hundreds of messages a day.
ROI
Risk and fraud detection systems save resources rather than generate new sources of income. But financial implications are vast – Deloitte predicted $40 billion in losses from fraud by 2027, in the US alone.
Simultaneously fraudsters are adopting GenAI and deep-learning models to improve their tactics. By quickly integrating automations that can act faster than humans, financial services organisations can strategically jump a few steps ahead.
One AI tool that can be repurposed to commit fraud by bypassing verification is video and audio deepfake technology.
RAG-based fraud detection systems help teams scan transactions, account access, and movements in real time, compare them to historical data, and analyse various accounts simultaneously to monitor for coordinated attacks.
As fraudsters tool up, RAG can push anti-fraud efforts to the offense and save banks and fintechs millions.
5. Intelligent customer service with RAG-based chatbots: AI in loan processing, 24/7 support, LLM augmenting
Previously, customer-facing chatbots in financial services have been run on legacy NLP (Natural Language Processing) models. NLPs could provide generalised information but answers often lacked relevance to a user’s specific account and situation.
Today, LLM-driven chatbots can initiate and host more natural, human-like chats. But in order to be high utility, these systems need to be able to retrieve personalised information securely.
The use of RAG models in banking and fintech is what enables systems to securely access internal data—account information, transaction history, location, etc. Consequentially, customers receive accurate, personalised answers according to their unique financial
situation.
One potential scenario is using LLM and RAG-powered chatbots for loan processing.
Rather than relying on a loan officer to manually collect the necessary documentation for a specific loan, RAG systems can automatically retrieve all required information like credit history and score.
Bottlenecks like lengthy applications and complex processing are removed, and customers receive application results and loans faster.
Considering that about 80% of retail customers mainly manage their banking digitally, the potential outcomes for AI in fintech customer service are clear.
Similar to investment research and market intelligence, automating parts of customer service also gives teams more time to navigate higher-value cases or more complex situations.
One financial chatbot using RAG is UK bank, NatWest’s chatbot Cora+. NatWest added IBM’s RAG-augmented GenAI to the basic version of their chatbot, and as a result Cora+ now personalises its answers based on variables like account location. More on this
in a moment.
ROI
ROI from the use of RAG models in banking and fintech chatbots is derived from a reduction in manual labor.
More interesting, though, is the influence 24/7 customer support and human-like language processing capabilities have on end customer satisfaction.
After integrating GenAI with their chatbot, NatWest reported customer satisfaction improved by 150%. Better first-contact resolution meant customer needs were met more quickly, reducing escalation and costs associated with further support.
Chatbots can even independently upsell or cross-sell products to end customers naturally as their account grows.
For example, it might recommend a better savings account based on a customer’s spending or saving patterns.
And an immediate, direct chat further removes friction by answering questions about why one option is superior according to their financial situation.
The bigger picture here being that quickly fulfilling customers’ needs improves customer retention.
Building and using high-ROI RAG models in banking and fintech
In November 2024, The Bank of England reported 75% of UK-based financial services were already using AI, and an added 10% plan to integrate AI within the next 3 years.
And in the EU, 40% of banks are already using general-purpose AI (GPAI); with the most popular use cases at 45% for customer support, and 40% for the optimisation of internal processes.
Competition means RAG applications in financial services are quickly expanding.
And they’re pushing past the ordinary RAG use cases like in lending, underwriting, KYC to higher ROI territory. Like J.P. Morgan and BBVA building agents with search functions tied to company-wide knowledge bases, and semi-automating investment and market
intelligence research.
But rather than deep pockets being the ultimate competitive differentiator, it is the architectural and data back end that will give organisations a leg up. A well-organised proprietary data repository is critical to RAG models running smoothly.
For financial services seriously considering RAG, a review of the entire back end is critical.
From there, leadership can select and implement the RAG use case that best fits their long-term goals and align returns with strategic priorities thanks to a system that actively learns and grows as their organisation does.
