TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?

In one sentence

Summary

This paper addresses the critical reliability gap in LLM-based autonomous trading agents, which are increasingly deployed in high-stakes financial environments but lack systematic robustness evaluation. The authors argue that while agent capability is well-studied, their faithfulness under adversarial or faulty conditions remains unexamined, posing significant risks in irreversible financial markets. The work positions itself as a stress-testing framework rather than a new agent architecture, focusing on the security and stability of existing adaptive and procedural agent designs. The proposed TradeTrap framework decomposes trading agents into four core components: market intelligence, strategy formulation, portfolio/ledger handling, and trade execution. It introduces six targeted attack modules, including data fabrication, MCP tool hijacking, prompt injection, memory poisoning, and state tampering. Experiments are conducted in a closed-loop historical backtesting setting on US equity data (NASDAQ-100) with identical initial conditions. The study evaluates both Adaptive agents (e.g., AI-Trader) and Procedural agents (e.g., NoFX, ValueCell) using nine quantitative metrics such as Total Return, Maximum Drawdown, and Position Concentration to measure the propagation of localized perturbations through the decision loop. Results show that small perturbations can induce extreme concentration, runaway exposure, and large drawdowns. Adaptive agents are highly sensitive to informational attacks (fake news, tool hijacking), leading to aggressive misallocation, while Procedural agents are more robust to noise but catastrophically fail when internal state is corrupted (memory poisoning, state tampering). For instance, state tampering caused a Procedural agent to suffer a -61% total return and -100% annualized return. The findings highlight a trade-off: adaptive agents capture more upside but are attackable via information channels, whereas procedural agents are structurally constrained but vulnerable to state corruption. The paper concludes that current agents lack system-level security and consistency checking.

Research problem

The paper investigates the reliability and robustness of LLM-based autonomous trading agents under adversarial or faulty conditions. It addresses the lack of systematic evaluation frameworks for stress-testing these agents against realistic failures that could propagate through the entire decision pipeline, from data ingestion to trade execution.

Core contributions

• Proposes TradeTrap, a unified evaluation framework for stress-testing autonomous trading agents across four functional components.
• Defines six specific attack modules targeting market intelligence, strategy formulation, and portfolio state management.
• Introduces a closed-loop backtesting protocol with identical initial conditions for fair comparison of adaptive and procedural agents.
• Demonstrates that small perturbations can cause catastrophic portfolio drawdowns and unreliable behavior in current agents.
• Reveals distinct vulnerability profiles: adaptive agents are sensitive to informational attacks, while procedural agents are vulnerable to state corruption.

Method

• Decomposes agents into market intelligence, strategy, portfolio, and execution components.
• Implements data fabrication by injecting coordinated fake news into external feeds.
• Simulates MCP tool hijacking by intercepting and substituting API responses with synthetic payloads.
• Applies prompt injection by inverting directional signals in decision prompts.
• Executes memory poisoning by appending fabricated transaction records to persistent position files.
• Performs state tampering by hooking position-reading interfaces to return false holding values.
• Measures impact using closed-loop backtesting with identical initial capital ($5,000) and market replay.
• Analyzes propagation of perturbations through decision trajectories and final portfolio metrics.

Results & findings

• Adaptive agents under fake news saw Sharpe ratio drop from 4.34 to 3.20 and total return from 11.59% to 5.26%.
• Prompt injection reduced Adaptive agent total return from 7.81% to 0.89% and Sharpe from 5.72 to 0.29.
• Memory poisoning caused Adaptive agent total return to drop from 7.81% to 1.88% and Calmar from 64.36 to 7.45.
• State tampering led to a Procedural agent total return of -61.02% and annualized return of -100.00%.
• Procedural agents showed lower volatility and concentration but suffered catastrophic losses when state was tampered.
• Adaptive agents exhibited 'strategic paralysis' and hallucination under MCP tool hijacking attacks.
• Fake news caused Adaptive agents to overreact with high position concentration (up to 79.66%).

Data & experiments

• Historical backtesting on US equity market data (NASDAQ-100) from October 1 to October 31.
• Evaluation of two agent architectures: Adaptive (AI-Trader) and Procedural (NoFX, ValueCell).
• Six attack scenarios: data fabrication, MCP tool hijacking, prompt injection, memory poisoning, state tampering.
• Nine quantitative metrics including Total Return, MDD, Sharpe Ratio, and Position Concentration.
• Single-variable intervention design to isolate causal effects of each attack module.

Limitations & caveats

• Experiments are limited to a single month (October) of historical data.
• Transaction fees and slippage are not included in the evaluation.
• Only single-asset or limited multi-asset scenarios are tested (NASDAQ-100 subset).
• Attacks are simulated and may not reflect all real-world adversarial capabilities.
• Does not evaluate multi-agent collaboration or complex market microstructure effects.
• Results may vary with different LLM backends or agent configurations.

Curator notes

Discovery status: trading_agent_focus_promoted. Source links: FinMem: A Performance-Enhanced Large Language Model Trading Agent with Layered Memory and Character Design