Large Language Model Agent in Financial Trading: A Survey

In one sentence

Summary

This paper addresses the emerging research area of using Large Language Models as autonomous agents in financial trading. It aims to map the landscape of LLM trading agents by answering three core questions: common architectures, data inputs, and current performance/limitations. The authors conducted a systematic review of 27 relevant papers, identifying this as the first comprehensive survey of LLM agents specifically for financial trading, distinguishing it from general LLM finance surveys by focusing on the agent paradigm and decision-making loops. The authors categorize agent architectures into two main types: 'LLM as a Trader,' which directly generates buy/sell/hold signals, and 'LLM as an Alpha Miner,' which uses LLMs to generate alpha factors for downstream quantitative systems. Within the trader category, they further classify approaches into news-driven, reflection-driven (using memory and reflection modules), debate-driven (multi-agent debate), and reinforcement learning-driven methods. Data inputs are analyzed across four types: numerical (prices, volumes), textual (fundamental reports, news, social media), visual (charts), and simulated environments. Experiments primarily involve backtesting on US and Chinese stock markets, using metrics like cumulative return, Sharpe ratio, and maximum drawdown, often comparing against rule-based or deep learning baselines. Key findings indicate that LLM agents generally outperform baselines in backtesting, achieving annualized returns of 15-30% over strong baselines. However, significant limitations exist: most studies rely on closed-source models, use short backtesting periods (median 1.3 years), ignore trading costs, and lack rigorous ablation studies on reasoning processes. The survey highlights that while LLMs excel at processing unstructured text, they struggle with numerical reasoning and high-frequency trading due to latency. Future directions include integrating multimodal data, exploring open-source models, and developing more robust evaluation frameworks that account for market frictions and ethical risks.

Research problem

The paper investigates the current state of Large Language Model (LLM) agents in financial trading, a rapidly evolving field where LLMs are increasingly used to automate trading decisions. It seeks to understand how these agents are architected, what data they consume, and how effectively they perform compared to traditional methods. The core problem is to synthesize fragmented research into a coherent framework to guide future development and identify critical gaps in methodology and evaluation.

Core contributions

• Provides the first comprehensive survey of LLM agents specifically for financial trading, reviewing 27 key papers.
• Proposes a hierarchical taxonomy of LLM trading agent architectures, distinguishing between 'LLM as Trader' and 'LLM as Alpha Miner'.
• Categorizes and analyzes the diverse data inputs used by agents, including numerical, textual, visual, and simulated data.
• Critically evaluates current backtesting methodologies, highlighting common flaws such as short time horizons and ignored trading costs.
• Identifies key limitations and future research directions, including the need for open-source models, multimodal integration, and robust ethical frameworks.

Method

• Systematic literature review of 27 papers on LLM agents in financial trading.
• Categorization of agent architectures into 'LLM as Trader' and 'LLM as Alpha Miner'.
• Sub-classification of trader agents into news-driven, reflection-driven, debate-driven, and RL-driven.
• Analysis of data inputs across numerical, textual, visual, and simulated categories.
• Evaluation of backtesting performance metrics including returns, Sharpe ratio, and drawdown.
• Assessment of LLM model usage and preference trends in the research community.

Results & findings

• LLM trading agents generally outperform baselines in backtesting, with annualized returns ranging from 15% to 30%.
• Reflection-driven and debate-driven architectures show improved robustness and reasoning capabilities.
• Multimodal agents incorporating visual data (charts) demonstrate superior performance over text-only counterparts.
• Most studies rely on short backtesting periods (median 1.3 years), raising concerns about result credibility.
• Trading costs are rarely considered in evaluations, potentially overstating agent profitability.
• Closed-source models dominate research, limiting reproducibility and customization.
• Social media data is underutilized despite its potential impact on market sentiment.

Data & experiments

• Systematic review of 27 papers identified via Google Scholar searches for 'LLM for trading' and 'GPT stock agent'.
• Analysis of data sources including financial reports (10-K/10-Q), news, social media, and market data.
• Evaluation of backtesting performance across US and Chinese stock markets, with some crypto experiments.
• Comparison of agent performance against rule-based, machine learning, and reinforcement learning baselines.
• Assessment of LLM model usage, noting a dominance of closed-source models like GPT-3.5 and GPT-4.

Limitations & caveats

• Most studies rely on closed-source models, limiting reproducibility and customization.
• Backtesting periods are often short (median 1.3 years), potentially overfitting to recent market conditions.
• Trading costs are rarely included in evaluations, leading to potentially inflated performance estimates.
• Lack of ablation studies on the reasoning processes of LLM agents.
• Limited exploration of multimodal data and social media data.
• Most evaluations are confined to US and Chinese stock markets, with little coverage of other asset classes.

Curator notes

Especially relevant for agentic trading systems.