The List / Agents and Multi-Agent Systems / paper
AlphaAgent: LLM-Driven Alpha Mining with Regularized Exploration to Counteract Alpha Decay
Abstract
Alpha mining, a critical component in quantitative investment, focuses on discovering predictive signals for future asset returns in increasingly complex financial markets. However, the pervasive issue of alpha decay-where factors lose their predictive power over time-poses a significant challenge for alpha mining. Traditional methods such as genetic programming are prone to rapid alpha decay, primarily due to their susceptibility to overfitting. At the same time, approaches driven by Large Language Models (LLMs), despite their promise, often fail to impose regularization against factor homogenization-resulting in crowded signals and accelerated decay. To address this challenge, we propose AlphaAgent, an autonomous framework that effectively integrates LLM-driven agents with ad hoc regularization for mining decay-resistant alpha factors. AlphaAgent employs three key mechanisms: (i) originality enforcement through a similarity measure based on abstract syntax trees (ASTs) against existing alphas(ii) hypothesis-factor alignment via LLM-evaluated semantic consistency between market hypotheses and generated factors, and (iii) complexity control via AST-based structural constraints, preventing over-engineered constructions that are prone to overfitting. These mechanisms collectively guide the alpha generation process to balance originality, financial rationale, and adaptability to evolving market conditions, mitigating the risk of alpha decay. Extensive evaluations show that AlphaAgent outperforms traditional and LLM-based methods in mitigating alpha decay across bull and bear markets, consistently delivering significant alpha in Chinese CSI 500 and U.S. S&P 500 markets over the past four years. Notably, AlphaAgent showcases remarkable resistance to alpha decay, elevating the potential for yielding powerful factors.
In one sentence
Summary
Alpha mining faces significant challenges from alpha decay, where predictive signals lose efficacy due to overfitting and factor crowding. Traditional genetic programming and reinforcement learning methods often generate complex, spurious factors that fail in live markets, while existing LLM-based approaches lack regularization, leading to homogenized signals that replicate known inefficiencies. This paper addresses the critical need for decay-resistant factor discovery by introducing a structured approach that balances novelty, financial rationale, and parsimony, aiming to sustain predictive power across evolving market regimes in both Chinese and U.S. equity markets. The proposed AlphaAgent framework employs a closed-loop multi-agent system comprising an idea agent, factor agent, and eval agent. The idea agent generates market hypotheses using chain-of-thought reasoning, while the factor agent constructs symbolic expressions using an operator library and abstract syntax trees (ASTs). Regularization mechanisms include AST-based similarity checks against existing alphas like Alpha101 to enforce originality, LLM-evaluated semantic consistency to ensure hypothesis alignment, and structural constraints to limit complexity. The system is evaluated on CSI 500 and S&P 500 data from 2021 to 2024, using LightGBM for return prediction and backtesting with transaction costs to assess performance. Experiments demonstrate that AlphaAgent outperforms traditional and LLM-based baselines, achieving an 81% higher hit ratio and significant annualized excess returns of 11.0% (IR=1.5) for CSI 500 and 8.74% (IR=1.05) for S&P 500. The framework shows remarkable resistance to alpha decay, maintaining stable predictive power across bull and bear markets. However, the study relies on historical data and specific LLM capabilities, and the effectiveness may vary with different base models or market conditions. The approach offers a robust pipeline for quantitative investment, highlighting the potential of regularized LLM agents in discovering sustainable alpha sources.
Research problem
Alpha mining in quantitative finance suffers from alpha decay, where factors lose predictive power over time due to overfitting and crowding. Traditional methods like genetic programming often produce complex, spurious factors, while LLM-based approaches lack regularization, leading to homogenized signals. The problem is to develop a method that generates novel, financially sound, and parsimonious alpha factors that resist decay.
Core contributions
- Proposes AlphaAgent, an autonomous multi-agent framework for LLM-driven alpha mining.
- Introduces three regularization mechanisms: originality enforcement via AST similarity, hypothesis alignment via LLM evaluation, and complexity control via structural constraints.
- Implements a closed-loop workflow with idea, factor, and eval agents for iterative factor refinement.
- Demonstrates superior performance and decay resistance in CSI 500 and S&P 500 markets over four years.
- Provides a systematic approach to balance factor novelty, financial rationale, and parsimony.
Method
- Idea agent generates market hypotheses using chain-of-thought reasoning.
- Factor agent constructs symbolic expressions using an operator library and ASTs.
- AST-based similarity check against Alpha101 to enforce originality.
- LLM-evaluated semantic consistency between hypothesis, description, and expression.
- Complexity control via symbolic length and parameter counting.
- Eval agent performs backtesting and provides feedback for iterative refinement.
- LightGBM model used for next-day return prediction with cross-sectional Z-score normalization.
- Top-k dropout strategy for stock selection in backtesting.
Results & findings
- AlphaAgent achieves 11.0% annualized excess return (IR=1.5) on CSI 500 and 8.74% (IR=1.05) on S&P 500.
- 81% improvement in effective factor ratio (hit ratio) compared to baselines.
- 30% fewer tokens consumed during factor generation compared to other LLM methods.
- Statistically significant improvements over RD-Agent with p-values < 0.05 for IC differences.
- DeepSeek-R1 as base LLM yields highest ICIR (0.0615) and lowest drawdown (-6.50%).
- Factors maintain stable predictive power across bull and bear markets, showing decay resistance.
- Outperforms traditional GP and RL methods in both performance and persistence.
Data & experiments
- Datasets: CSI 500 (Chinese A-shares) and S&P 500 (U.S. stocks) from 2021 to 2024.
- Features: OHLCV data (open, high, low, close, volume) used to construct factors.
- Baselines: LSTM, Transformer, LightGBM, TRA, StockMixer, AlphaForge, RD-Agent, OpenAI-o1, DeepSeek-R1.
- Metrics: IC, RankIC, ICIR, AR, IR, MDD, and hit ratio.
- Backtesting: Qlib framework with transaction costs and top-k dropout strategy.
- Evaluation: Comparison of performance across different market regimes and LLM variants.
Limitations & caveats
- Relies on historical data, which may not fully capture future market dynamics.
- Performance depends on the quality of the base LLM used in the agents.
- Transaction costs are simplified and may not reflect real-world trading frictions.
- The framework is tested on CSI 500 and S&P 500, limiting generalizability to other markets.
- AST-based similarity may not capture semantic similarity in all cases.
- Hypothesis generation relies on initial human insights, which may be biased.
Curator notes
Discovery status: institutional_trading_round2_promoted. Source links: From Natural Language to Executable Option Strategies via Large Language Models