Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challenges

In one sentence

Summary

The paper addresses the gap in general-purpose foundation models regarding the unique domain requirements of financial engineering, such as multimodal reasoning, regulatory compliance, and data privacy. It introduces a comprehensive taxonomy of Financial Foundation Models (FFMs), distinguishing between Financial Language Foundation Models (FinLFMs), Financial Time-Series Foundation Models (FinTSFMs), and Financial Visual-Language Foundation Models (FinVLFMs). The authors aim to provide a unified view of the FFM landscape, covering progress from 2018 to 2025, and serve as a reference for researchers and practitioners navigating this rapidly evolving field. The methodology involves a systematic literature review and categorization of 21 representative FinLFMs, along with an overview of emerging FinTSFMs and FinVLFMs. The authors analyze training pipelines including pre-training, supervised fine-tuning, and alignment techniques like reinforcement learning for reasoning. They survey a wide range of datasets, from early task-specific English corpora to recent multi-task, cross-lingual benchmarks like AlphaFin and FinBen. The review also examines real-world applications, including sentiment analysis, stock prediction, and regulatory compliance checking, while highlighting the shift from BERT-style encoders to GPT-style generative models and reasoning-enhanced agents. Key findings indicate that FinLFMs have matured significantly through instruction tuning and alignment, with models like BloombergGPT and FinGPT demonstrating strong domain adaptation. However, FinTSFMs remain in early stages due to the non-stationary nature of financial time series. The paper identifies critical challenges in data availability, algorithmic scalability, and infrastructure constraints. It emphasizes the need for robust evaluation benchmarks and regulatory-compliant alignment. The survey concludes by offering a roadmap for future research, stressing the importance of multimodal integration and autonomous financial agents, while noting that current models still struggle with complex temporal dynamics and strict privacy requirements.

Research problem

General-purpose foundation models often fail to meet the specific needs of financial engineering, such as handling multimodal data (text, time-series, charts), ensuring regulatory compliance, and maintaining data privacy. There is a lack of a unified framework to understand the diverse landscape of Financial Foundation Models (FFMs) and their specific architectural and training requirements. This paper seeks to systematize the development of FFMs, categorizing them by modality and reviewing their progress, challenges, and future directions.

Core contributions

• Proposes a systematic taxonomy of FFMs into three modalities: language (FinLFMs), time-series (FinTSFMs), and visual-language (FinVLFMs).
• Provides a comprehensive review of 21 representative FinLFMs, detailing their architectures, training strategies, and evolution from BERT-style to reasoning-enhanced models.
• Surveys a wide range of financial datasets and benchmarks, categorizing their evolution from task-specific English corpora to multi-task, cross-lingual resources.
• Analyzes training methodologies, including pre-training, supervised fine-tuning, and alignment techniques like reinforcement learning for reasoning and compliance.
• Identifies critical challenges in data availability, algorithmic scalability, and infrastructure constraints, offering insights into future research opportunities.

Method

• Systematic literature review of FFM developments from 2018 to 2025.
• Categorization of models by input modality: language, time-series, and visual-language.
• Analysis of training pipelines: pre-training, supervised fine-tuning, and alignment.
• Review of dataset evolution and characteristics across different financial NLP tasks.
• Comparison of existing surveys to highlight the comprehensiveness of this work.
• Identification of challenges in data, algorithms, and infrastructure.
• Synthesis of real-world applications and deployment scenarios.

Results & findings

• FinLFMs have evolved from BERT-style encoders to GPT-style chatbots and reasoning-enhanced agents.
• BloombergGPT demonstrates the feasibility of domain-specific LLMs at scale using proprietary data.
• Recent trends favor continuous pre-training on open-source base models (e.g., LLaMA, Qwen) rather than training from scratch.
• Alignment techniques, including reinforcement learning, are crucial for ensuring factual accuracy and regulatory compliance.
• FinTSFMs remain in early stages due to the non-stationary and multivariate nature of financial time series.
• FinVLFMs complement FinTSFMs by handling visual-symbolic layers like charts and tables.
• Datasets are expanding to include multi-task, cross-lingual, and instruction-tuned resources for better benchmarking.
• Key challenges include data privacy, algorithmic scalability, and the need for standardized evaluation benchmarks.

Data & experiments

• Reviews 21 representative FinLFM models, including BloombergGPT, FinGPT, and XuanYuan2.
• Surveys multiple financial text-based datasets such as FPB, FiQA, AlphaFin, and FinBen.
• Categorizes datasets by evolution stage: early task-specific, multi-task integration, and recent cross-lingual/real-world benchmarks.
• Discusses emerging FinTSFM and FinVLFM architectures and their associated data requirements.
• Evaluates models based on training stages: pre-training, supervised fine-tuning, and alignment.
• Notes that the paper is a survey and does not present new experimental results but synthesizes existing literature.

Limitations & caveats

• As a survey, it does not present new experimental results or empirical evaluations.
• The rapid evolution of the field means some models or datasets may become outdated quickly.
• The survey focuses on English and Chinese datasets, potentially overlooking other linguistic contexts.
• FinTSFMs and FinVLFMs are less mature than FinLFMs, limiting the depth of discussion in those areas.
• The paper does not provide a detailed comparison of model performance across different benchmarks.
• Challenges in data privacy and regulatory compliance are identified but not fully resolved.

Curator notes

Discovery status: round1_promoted_seed_for_round2. Source links: A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges; A Survey of Large Language Models in Finance: FinLLMs; A Survey on Large Language Models for Critical Societal Domains: Finance, Healthcare, and Law; AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Framework; Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment Analysis; BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-tra