A Review on Large Language Models and Generative AI in Banking

In one sentence

Summary

This paper addresses the critical intersection of Generative AI and the banking sector, a domain characterized by high potential for efficiency gains but also by strict regulatory requirements and high stakes for accuracy. The authors conduct a Systematic Literature Review (SLR) to answer the research question regarding the current state of incorporating GenAI and LLMs in banking. The study aims to identify common study types, specific tasks where LLMs are explored, and the apparent challenges and concerns emerging from the scientific literature. It positions itself as a foundational overview in a rapidly evolving field, noting that while many papers offer general overviews, few provide structured, rigorous reviews of the domain's specific application landscape. The methodology involves a rigorous SLR protocol across four major scientific databases: Scopus, IEEE Xplore, ACM Digital Library, and AIS Electronic Library. The search query combined terms for GenAI/LLMs with banking-related terms, restricted to titles, and filtered for peer-reviewed journal articles and conference papers in English, explicitly excluding preprints and papers focused on auxiliary activities like stock trading or document analysis. The final corpus comprises 14 papers published in 2023 and 2024. The analysis categorizes these papers by research type (overview vs. specific development), addressed area, tasks mentioned, challenges, models used, and prompting strategies. Key experiments reviewed include prompt engineering for credit risk reports, multi-agent architectures for digital banking assistants, LLM-based data augmentation for bankruptcy prediction, and fairness studies for financial advisement. The findings reveal that most research is experimental, with no identified papers having deployed LLMs productively in core banking tasks at the time of writing. Common applications include customer service, credit risk analysis, fraud detection, and regulatory compliance. Significant challenges persist, including hallucinations, data privacy, biases, lack of transparency, and insufficient model quality for critical decisions. The review highlights that language-specific models often outperform general ones like ChatGPT in non-English contexts. A major limitation is the lack of standardized reporting in LLM projects, making reproducibility difficult. The paper concludes that while the potential is vast, the technology is in its infancy regarding banking integration, requiring further maturity and robust evaluation frameworks before widespread adoption.

Research problem

The paper investigates the current state of incorporating Generative AI and Large Language Models into the banking sector. It seeks to understand which types of studies are common, for which specific banking tasks LLMs are being explored, and what challenges and concerns have become apparent in the scientific literature. The problem is framed by the tension between the theoretical benefits of GenAI for processing high volumes of banking data and the strict regulatory and accuracy demands of the industry.

Core contributions

• Conducts a rigorous Systematic Literature Review (SLR) across four major databases to map the landscape of LLM applications in banking.
• Identifies and categorizes 14 key papers from 2023-2024, providing a structured overview of research types and trends.
• Synthesizes common application scenarios, including customer service, credit risk, fraud detection, and regulatory compliance.
• Highlights the gap between experimental research and productive deployment, noting the absence of real-world production cases in the reviewed literature.
• Identifies key challenges such as hallucinations, data privacy, biases, and the superiority of language-specific models over general ones.
• Critiques the lack of standardized reporting in LLM projects, noting missing details on prompts, models, and evaluation metrics.

Method

• Systematic Literature Review (SLR) following established protocols for rigor and reproducibility.
• Search strategy using specific keywords for GenAI/LLMs and banking terms, restricted to document titles.
• Filtering process based on inclusion/exclusion criteria: peer-reviewed, English, core banking focus, excluding trading/document analysis.
• Data extraction and categorization of papers by research type, addressed area, tasks, challenges, models, and prompting.
• Comparative analysis of model performance and prompting strategies across different studies.
• Synthesis of findings to identify common trends, challenges, and future research directions.

Results & findings

• The final literature set comprises 14 papers, all published in 2023 or 2024, indicating the recency of the field.
• Most papers are overviews or specific developments; none represent productive, real-world deployments in core banking tasks.
• Customer service and financial advisement are the most frequently mentioned application areas.
• LLM-generated credit risk reports were often preferred by human analysts over human-generated ones in blinded evaluations.
• Language-specific LLMs outperformed general models like ChatGPT in non-English contexts, suggesting a need for specialized models.
• Direct LLM prediction of bankruptcy was found to be insufficient, though LLMs were useful for data augmentation.
• Significant concerns were raised regarding hallucinations, data privacy, biases, and the lack of transparency in LLM outputs.
• No standardized reporting practices were found, with many papers omitting crucial details like prompt strategies and model versions.

Data & experiments

• Systematic Literature Review of 14 papers from Scopus, IEEE Xplore, ACM Digital Library, and AIS Electronic Library.
• Papers selected from 2023 and 2024, focusing on core banking activities and excluding stock trading or document analysis.
• Analysis of specific experimental studies including prompt engineering for credit reports, multi-agent banking assistants, and bankruptcy prediction using data augmentation.
• Evaluation of model performance comparisons, such as GPT-3.5 vs. GPT-4 vs. Bard, and general vs. language-specific models.
• Assessment of fairness and efficacy in financial advisement tasks, including bias detection based on user demographics.

Limitations & caveats

• The review is limited to papers published in 2023 and 2024, potentially missing earlier foundational work.
• It excludes preprints, which may contain timely but unreviewed research in this fast-moving field.
• The scope explicitly excludes stock trading and document analysis, limiting its relevance for those specific subfields.
• The lack of real-world deployment studies in the corpus limits the ability to assess practical implementation challenges.
• The heterogeneity of the 14 included papers makes direct comparison difficult.
• The review relies on the quality and reporting standards of the included papers, which were found to be inconsistent.

Curator notes

Discovery status: regtech_compliance_focus_promoted. Source links: LLMs for Financial Advisement: A Fairness and Efficacy Study in Personal Decision Making