A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist

In one sentence

Summary

Financial trading requires processing diverse, multimodal data and adapting to rapid market dynamics, yet existing AI agents often struggle with generalizability and the integration of visual and textual information. The authors address these gaps by proposing FinAgent, a multimodal foundation agent designed to handle numerical, textual, and visual market intelligence. The system aims to bridge the gap between simple question-answering LLMs and complex sequential decision-making in trading by incorporating reasoning, memory, and tool use to enhance trust and adaptability in volatile environments. FinAgent employs a five-module architecture: a market intelligence module for summarizing news and prices, a diversified memory retrieval system to store and fetch relevant historical insights, and a dual-level reflection module. The low-level reflection analyzes price movements against market intelligence, while the high-level reflection evaluates past trading decisions to learn from successes and mistakes. The decision-making module integrates these insights with expert guidance and technical indicators (e.g., MACD, KDJ) to generate buy, sell, or hold actions. Experiments were conducted on six financial datasets, including stocks and cryptocurrencies, comparing FinAgent against 12 state-of-the-art baselines such as FinGPT, FinMem, and reinforcement learning methods like SAC and DQN. FinAgent achieved an average profit improvement of over 36% across six financial metrics and secured a 92.27% return on one dataset, significantly outperforming all baselines. The agent demonstrated superior ability to capitalize on market trends and avoid losses compared to models that failed to adapt to volatility or misinterpreted news. However, limitations include the reliance on specific prompt engineering for reasoning, potential sensitivity to the quality of retrieved historical data, and the computational overhead of multimodal processing. The study highlights the value of structured reflection and tool augmentation in enhancing LLM-based trading agents.

Research problem

Existing AI agents for financial trading often fail to adequately process multimodal data (text, numbers, charts) and lack generalizability across different assets and market conditions. Traditional reinforcement learning methods are data-hungry and opaque, while current LLM agents struggle with the sequential nature of trading and integrating domain-specific tools. This paper addresses the need for a generalist, multimodal agent that can reason, reflect, and adapt in real-time financial environments.

Core contributions

• Introduction of FinAgent, a multimodal foundation agent for financial trading that processes news, prices, and Kline charts.
• Design of a diversified memory retrieval system that separates query generation for trading vs. retrieval to reduce noise.
• Development of a dual-level reflection module: low-level for price-intelligence correlation and high-level for decision evaluation.
• Integration of tool-augmented decision-making with expert guidance and technical indicators to infuse domain knowledge.
• Comprehensive evaluation on six datasets (stocks and crypto) showing significant profit improvements over 12 baselines.

Method

• Market Intelligence Module: Summarizes latest news, prices, and reports into insights and query texts.
• Diversified Retrieval: Uses specific query fields to retrieve relevant past market intelligence from vector memory.
• Low-Level Reflection: Analyzes correlation between market intelligence/technical indicators and price changes.
• High-Level Reflection: Evaluates past trading actions and reasoning to identify improvements.
• Decision-Making Module: Integrates reflections, expert guidance, and trader preferences to output Buy/Sell/Hold.
• Tool Augmentation: Utilizes technical indicators (MACD, KDJ, RSI) and expert strategies as external tools.

Results & findings

• FinAgent outperforms 12 baselines with over 36% average improvement in profit across six financial metrics.
• Achieved a 92.27% return on one specific dataset, representing an 84.39% relative improvement.
• FinGPT showed limitations in capitalizing on peaks and making wrong predictions on sharp drops.
• SAC and DQN struggled with market volatility and timing entry/exit points in high-fluctuation environments.
• FinMem exhibited unstable decision-making and failed to account for cash constraints in some scenarios.
• FinAgent's reflection module effectively learned from past mistakes, improving future decision quality.

Data & experiments

• Experiments conducted on six financial datasets including stocks (e.g., AAPL, MSFT, TSLA) and cryptocurrencies (e.g., ETHUSD).
• Comparison against 12 state-of-the-art baselines, including rule-based strategies, RL methods (SAC, DQN), and LLM agents (FinGPT, FinMem).
• Evaluation metrics include total profit, Sharpe ratio, and other standard financial performance indicators.
• Analysis of specific failure cases in baselines, such as missing peaks or incorrect predictions on sharp price drops.

Limitations & caveats

• Relies on the quality of the underlying multimodal LLM and prompt engineering.
• Performance may vary with the quality and relevance of retrieved historical data.
• Computational overhead of processing multimodal data and running reflection modules.
• Potential for overfitting to specific market conditions or assets in the training/evaluation set.
• Does not explicitly address transaction costs or slippage in all scenarios.

Curator notes

Discovery status: round1_promoted_seed_for_round2. Source links: A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges; Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews; AlphaQuanter: An End-to-End Tool-Augmented Agentic Reinforcement Learning Framework for Stock Trading; Automate Strategy Finding with LLM in Quant investment; Beyond Isolated Investor: Predicting Startup Success via Roleplay-Based Collective Agents; BloombergGPT: A Large Language Model for Finance; C