Large Language Models in Finance: A Survey

In one sentence

Summary

The paper addresses the growing intersection of large language models and financial applications, aiming to provide a practical roadmap for researchers and practitioners. It positions LLMs as transformative tools for tasks ranging from trading and risk modeling to customer service, emphasizing their ability to leverage pretraining knowledge, perform complex reasoning, and orchestrate multi-agent tool use. The survey covers the evolution from traditional AI to LLMs, detailing how zero-shot, few-shot, fine-tuning, and from-scratch pretraining approaches offer varying degrees of domain adaptation and performance. It specifically highlights the trade-offs between using proprietary APIs, open-source models, and custom-trained systems, noting that while LLMs excel in flexibility and common-sense reasoning, they require careful selection based on data availability, compute resources, and privacy constraints. The methodology involves a comprehensive review of existing literature and case studies, categorizing LLM solutions into four levels of complexity: zero-shot, few-shot, tool-augmented/fine-tuned, and from-scratch training. The authors evaluate specific models such as FinMA, FinGPT, BloombergGPT, and Fin-T5, analyzing their performance on financial classification tasks (e.g., sentiment analysis, news headline classification) and generative tasks (e.g., question answering, summarization). Experiments cited in the survey demonstrate that fine-tuned and pre-trained finance-specific models significantly outperform general-purpose LLMs in classification tasks, with BloombergGPT achieving an average score of 62.51 compared to 54.35 for BLOOM176B. The paper also provides a decision framework that maps use-case constraints to appropriate LLM strategies, including cost estimates for development and deployment, helping practitioners navigate the balance between performance and resource investment. Key findings indicate that while fine-tuned finance LLMs show superior performance in classification, their generative capabilities often lag behind general models like GPT-4, suggesting a need for higher-quality domain datasets. The survey identifies critical limitations, including the risk of hallucination, bias, and regulatory challenges, which necessitate safeguards like retrieval-augmented generation and content censorship. It concludes that LLMs are most valuable for tasks requiring reasoning, orchestration, or handling unstructured data where labeled examples are scarce. The paper serves as a foundational reference for understanding the current landscape, offering concrete guidance on model selection, evaluation metrics, and ethical considerations, while acknowledging that for simple, well-defined tasks with ample data, traditional machine learning may remain more cost-effective and reliable.

Research problem

The paper addresses the lack of a unified, practical guide for applying large language models in the finance industry. It seeks to synthesize the diverse approaches to LLM adoption, from zero-shot inference to custom pretraining, and provide a structured decision framework for financial professionals to select appropriate solutions based on constraints like data, compute, and performance needs. The problem also encompasses understanding the specific capabilities and limitations of LLMs in financial contexts, such as handling unstructured text, reasoning, and risk management.

Core contributions

• Provides a comprehensive survey of LLM applications in finance, covering trading, risk modeling, text mining, and advisory services.
• Reviews and compares various LLM solution strategies, including zero-shot, few-shot, fine-tuning, and from-scratch pretraining.
• Proposes a four-level decision framework to guide practitioners in selecting the optimal LLM strategy based on resource constraints.
• Evaluates the performance of key finance-specific LLMs like FinMA, FinGPT, BloombergGPT, and Fin-T5 on classification and generative tasks.
• Discusses critical limitations and risks, including hallucination, bias, and regulatory challenges, offering mitigation strategies.
• Offers cost estimates and practical guidance for development and deployment of LLMs in financial settings.

Method

• Conducts a systematic literature review of LLM applications in finance, categorizing by task and model type.
• Analyzes existing finance-specific LLMs, detailing their training data, architecture, and evaluation results.
• Proposes a decision framework that maps use-case constraints to four levels of LLM adoption complexity.
• Evaluates model performance using standard NLP metrics (F1, Precision, Recall) and financial performance metrics.
• Compares proprietary and open-source models, assessing trade-offs in privacy, cost, and performance.
• Reviews techniques for fine-tuning, including instruction tuning, LoRA, and quantization, for resource efficiency.
• Discusses the role of tool-augmented generation and multi-agent systems in enhancing LLM capabilities.
• Identifies limitations and risks, proposing safeguards like retrieval-augmented generation and content censorship.

Results & findings

• Fine-tuned finance LLMs significantly outperform general open-source models in financial classification tasks.
• BloombergGPT achieves an average score of 62.51 on finance benchmarks, surpassing BLOOM176B's 54.35.
• Fin-T5 outperforms T5 with an average score of 81.78 compared to 79.56 on similar tasks.
• Fine-tuned models show similar or worse performance than GPT-4 in generative tasks, indicating a need for better data.
• Even a small subset of private finance data (0.7% of BloombergGPT's corpus) leads to substantial performance gains.
• LLMs excel in tasks requiring reasoning and orchestration, making them suitable for complex financial advisory.
• Zero-shot and few-shot learning offer low-cost entry points but may lack optimal performance for specialized tasks.
• Tool-augmented generation and fine-tuning are necessary for complex tasks where in-context learning is insufficient.

Data & experiments

• Reviews performance metrics of fine-tuned models like FinMA, FinGPT, and Instruct-FinGPT on financial sentiment analysis and news classification.
• Analyzes BloombergGPT and Fin-T5 performance on internal and external benchmarks, comparing them to general models like BLOOM and T5.
• Cites experimental results showing fine-tuned models outperform general LLMs in classification but lag in generative tasks.
• Provides cost analysis tables for different LLM deployment options, including API costs and computational resources.
• Evaluates models on accuracy metrics (F1, Precision, Recall) and performance metrics (Sharpe Ratio, total profit) where applicable.
• Discusses the use of internal benchmarks by Bloomberg for evaluating BloombergGPT's domain-specific capabilities.

Limitations & caveats

• Fine-tuned finance LLMs may underperform general models in generative tasks, indicating a need for better data.
• The survey relies on reported results from other studies, which may not be directly comparable or reproducible.
• Cost estimates are approximate and may vary based on specific use cases and provider pricing.
• The decision framework is general and may not account for all industry-specific regulatory requirements.
• LLMs are prone to hallucination and bias, which can lead to inaccurate or unfair financial decisions.
• The rapid evolution of LLMs means that some models and techniques discussed may become outdated quickly.

Curator notes

Covers prompting, fine-tuning, custom financial LLMs, and institutional adoption considerations.