When Agents Trade: Live Multi-Market Trading Arena for LLM Agents

In one sentence

Summary

The paper addresses the limitation of static backtesting in financial LLM research by introducing the Agent Trading Arena, a virtual zero-sum stock market where LLM-based agents engage in competitive trading. Unlike traditional methods where agents cannot influence prices, this platform simulates real-time bid-ask interactions, allowing agents to directly impact price dynamics and experience market friction, liquidity constraints, and slippage. This closed-loop design bridges the gap between training and evaluation by forcing agents to adapt to shifting market conditions and opponent behaviors rather than relying on historical patterns. The proposed method, ArenaTrader, integrates textual and visual modalities, using line charts and bar graphs alongside price data to enhance numerical reasoning. A reflection module is incorporated to enable iterative strategy refinement by evaluating past trades and contrasting successful with unsuccessful tactics. Experiments were conducted on NASDAQ and CSI datasets, comparing ArenaTrader against baselines like Buy & Hold, SMA, MACD, and deep learning models such as StockFormer and TimesNet. The evaluation focused on total return, Sharpe ratio, and win rate under varying volatility regimes. Results indicate that LLMs struggle with numerical reasoning on plain-text data, often overfitting to local patterns, whereas visual inputs substantially improve performance. The ArenaTrader with GPT-4o achieved a Sharpe ratio of 0.348 on NASDAQ, outperforming the benchmark and all baselines. The reflection module further boosted returns, particularly with visual inputs. The study highlights that while LLMs show promise in dynamic environments, they remain sensitive to input modality and require mechanisms like reflection to handle complex, interactive financial scenarios effectively.

Research problem

Existing LLM applications in finance rely heavily on static backtesting, which fails to capture the interactive nature of real markets where trading actions influence prices. This paper addresses the need for a dynamic evaluation framework that simulates real-time market feedback, liquidity constraints, and multi-agent competition to better assess LLMs' adaptability and reasoning capabilities in financial decision-making.

Core contributions

• Design of the Agent Trading Arena, a closed-loop, zero-sum multi-agent simulation that enables real-time price formation and agent interaction.
• Demonstration that LLMs struggle with numerical reasoning on textual financial data but perform significantly better with visual inputs like charts.
• Integration of a reflection module that iteratively refines trading strategies, leading to improved profitability and stability.
• Empirical validation on NASDAQ and CSI datasets showing superior performance of the proposed method over traditional technical and deep learning baselines.

Method

• Closed-loop simulation with bid-ask mechanism for price formation.
• Multi-modal input: Textual price data combined with visual charts (line, bar, scatter).
• Reflection module: Daily evaluation of trading trajectories and strategy optimization.
• Zero-sum competition: Agents compete against each other, influencing market dynamics.
• Chat Pool: Shared communication channel for rumors and market sentiment.
• Wealth fee and dividends: Incentives to encourage active trading and prevent passive holding.

Results & findings

• GPT-4o achieved the highest total return (47.69%) and Sharpe ratio (6.777) with visual inputs and reflection.
• Visual inputs significantly outperformed textual inputs across all tested LLMs.
• The reflection module improved GPT-4o's total return by 41.7% compared to text-only without reflection.
• ArenaTrader outperformed deep learning baselines like StockFormer and TimesNet on both NASDAQ and CSI.
• LLMs showed difficulty with percentage changes and long-range dependencies in textual data.
• The Arena successfully simulated market microstructure effects like slippage and liquidity constraints.

Data & experiments

• Datasets: NASDAQ (2-month simulation) and CSI (1-year simulation) from Yahoo Finance and Baostock.
• Baselines: Buy & Hold, SMA, ZMR, MACD, StockFormer, TimesNet, StockMixer.
• Metrics: Total Return, Win Rate, Sharpe Ratio, Mean Daily Return, Return Volatility.
• Experiments: Comparison of textual vs. visual inputs, impact of reflection module, and performance across different LLM backbones (GPT-4o, Gemini-1.5, Qwen-VL).
• Ablation studies on input window sizes and modality combinations.

Limitations & caveats

• The simulation is a simplified zero-sum environment, which may not fully capture real-world market complexity.
• LLMs' performance is highly dependent on the quality of visual inputs and the reflection module.
• The study focuses on short-term trading strategies, limiting generalizability to long-term investment.
• Potential for overfitting to the specific simulation dynamics rather than real market conditions.
• The impact of external news and macroeconomic factors is simulated rather than real.

Curator notes

Discovery status: trading_agent_focus_promoted. Source links: FinMem: A Performance-Enhanced Large Language Model Trading Agent with Layered Memory and Character Design