Interpretable LLMs for Credit Risk: A Systematic Review and Taxonomy

In one sentence

Summary

The paper addresses the lack of systematic classification in the rapidly growing field of Large Language Models (LLMs) applied to credit risk assessment. While traditional credit scoring relies on structured financial ratios, LLMs offer the potential to extract risk signals from unstructured texts like analyst reports, disclosures, and news. The authors identify a critical gap in existing literature: most surveys either focus broadly on generative AI in banking or narrowly on specific ML techniques, failing to provide a comprehensive taxonomy that integrates model architectures with the crucial requirement of interpretability (XAI) for high-stakes financial decisions. This work aims to serve as a reference framework for researchers and practitioners navigating the intersection of NLP, finance, and explainable AI. The methodology follows the PRISMA guidelines to select 60 relevant papers published between 2020 and 2025 from major academic databases. The authors construct a four-pillar taxonomy: model architectures (encoder-only, decoder-only, hybrid, and domain-specific FinLLMs), data modalities (structured, unstructured text, time-series, multimodal, and synthetic data), interpretability mechanisms (post-hoc XAI like SHAP/LIME, chain-of-thought prompting, and intrinsically transparent designs), and application areas (retail/SME scoring, fraud detection, sentiment analysis, and investment). Experiments and findings are synthesized from the selected studies, covering benchmarks like FinLLM Leaderboard and FinLMEval, and evaluating metrics such as F1 scores, fairness indices (ISIP/ISA), and hallucination rates. The review highlights the shift from black-box predictions to transparent, regulation-compliant systems. Key findings indicate that encoder-only models (e.g., FinBERT) excel in fine-tuning for classification, while decoder models (e.g., GPT-4) are preferred for generative tasks and few-shot scenarios. Hybrid pipelines combining LLMs with traditional ML (e.g., GPT-LGBM) and Retrieval-Augmented Generation (RAG) show promise in mitigating data scarcity and hallucination. Interpretability is achieved through post-hoc methods, instruction tuning, and novel architectures like Logit Leaf. Use cases span P2P lending, SME credit scoring, AML, and bond yield prediction. Limitations include the predominance of preprints, lack of standardized benchmarks for credit-specific XAI, and challenges in ensuring fairness and reproducibility across diverse datasets. The paper concludes by outlining future directions for robust, interpretable, and fair LLM deployment in credit risk.

Research problem

The paper identifies the absence of a systematic review and taxonomy for LLM-based credit risk assessment. It addresses the need to categorize diverse LLM architectures, data sources, and explainability mechanisms, which are critical for regulatory compliance and trust in financial decision-making. The problem is further compounded by the superficial treatment of interpretability in existing surveys.

Core contributions

• First systematic review and taxonomy of LLM-based credit risk assessment using PRISMA methodology.
• Four-pillar taxonomy covering model architectures, data modalities, interpretability mechanisms, and application areas.
• Critical analysis of 60 papers (2020-2025) to identify trends, challenges, and research gaps.
• Synthesis of interpretability techniques (XAI, CoT, intrinsic transparency) specific to credit modeling.
• Identification of future research directions for robust, fair, and reproducible LLM deployment in finance.

Method

• PRISMA-based systematic literature search across multiple academic databases.
• Keyword-based filtering for LLMs, credit risk, explainability, and financial NLP.
• Snowball sampling to expand the initial set of 51 papers to 60.
• Four-dimensional taxonomy construction: architecture, data, interpretability, application.
• Qualitative synthesis of model performance, data types, and XAI mechanisms.
• Critical analysis of limitations, biases, and regulatory challenges in existing studies.
• Comparison with prior surveys to highlight the novelty of the proposed taxonomy.

Results & findings

• Encoder-only models (e.g., FinBERT) show superior fine-tuning performance for classification tasks.
• Decoder models (e.g., GPT-4) excel in few-shot and generative tasks but face hallucination risks.
• Hybrid pipelines (LLM + traditional ML) and RAG improve robustness and mitigate data scarcity.
• Post-hoc XAI (SHAP/LIME) is common, but intrinsic transparency and CoT are emerging trends.
• Synthetic data generation (e.g., FinGPT) helps address class imbalance and data privacy issues.
• Fairness and bias mitigation are critical but under-standardized in current credit risk LLM applications.
• Domain-specific FinLLMs (e.g., ZiGong, FinMA) outperform general models in specialized credit tasks.
• Regulatory compliance requires robust interpretability, driving the shift from black-box to transparent models.

Data & experiments

• Systematic literature review of 60 papers selected via PRISMA from IEEE, Elsevier, ACM, Scopus, and arXiv.
• Analysis of model architectures including encoder-only (FinBERT), decoder-only (GPT-4), and hybrid pipelines.
• Evaluation of data modalities: structured financial data, unstructured text (news, filings), time-series, and synthetic data.
• Review of interpretability methods: SHAP, LIME, Chain-of-Thought, and intrinsically transparent models.
• Assessment of application domains: SME credit scoring, fraud detection, sentiment analysis, and investment.
• Discussion of benchmarks like FinLLM Leaderboard and FinLMEval for standardized evaluation.

Limitations & caveats

• Many included studies are preprints, which may lack rigorous peer review.
• Lack of standardized benchmarks for credit-specific interpretability and fairness.
• Potential bias in selected papers towards English-language publications.
• Rapidly evolving field means some models and techniques may become outdated quickly.
• Limited discussion on real-world deployment challenges and computational costs.
• Synthesis is qualitative; lacks meta-analysis of quantitative performance metrics across studies.

Curator notes

Discovery status: round3_promoted_seed_for_round4. Source links: A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law; Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models; LLMs for Financial Advisement: A Fairness and Efficacy Study in Personal Decision Making; Large Language Models for Financial and Investment Management: Applications and Benchmarks