The List / Trading, Investment, and Portfolio Management / paper
CN-Buzz2Portfolio: A Chinese-Market Dataset and Benchmark for LLM-Based Macro and Sector Asset Allocation from Daily Trending Financial News
Abstract
Large Language Models (LLMs) are rapidly transitioning from static Natural Language Processing (NLP) tasks including sentiment analysis and event extraction to acting as dynamic decision-making agents in complex financial environments. However, the evolution of LLMs into autonomous financial agents faces a significant dilemma in evaluation paradigms. Direct live trading is irreproducible and prone to outcome bias by confounding luck with skill, whereas existing static benchmarks are often confined to entity-level stock picking and ignore broader market attention. To facilitate the rigorous analysis of these challenges, we introduce CN-Buzz2Portfolio, a reproducible benchmark grounded in the Chinese market that maps daily trending news to macro and sector asset allocation. Spanning a rolling horizon from 2024 to mid-2025, our dataset simulates a realistic public attention stream, requiring agents to distill investment logic from high-exposure narratives instead of pre-filtered entity news. We propose a Tri-Stage CPA Agent Workflow involving Compression, Perception, and Allocation to evaluate LLMs on broad asset classes such as Exchange Traded Funds (ETFs) rather than individual stocks, thereby reducing idiosyncratic volatility. Extensive experiments on nine LLMs reveal significant disparities in how models translate macro-level narratives into portfolio weights. This work provides new insights into the alignment between general reasoning and financial decision-making, and all data, codes, and experiments are released to promote sustainable financial agent research.
In one sentence
Summary
The paper addresses the evaluation gap in financial LLM agents, arguing that existing benchmarks are either irreproducible live trading systems or entity-centric static tests that ignore public attention dynamics. It introduces CN-Buzz2Portfolio, a reproducible benchmark for the Chinese market that maps daily trending news to macro and sector asset allocation, aiming to isolate reasoning logic from stochastic market noise. The authors posit that current paradigms fail to test the 'narrative sifting' process required in real-world trading, where investors must identify relevant sectors from unstructured trending topics rather than pre-filtered entity news. The methodology employs a Tri-Stage CPA Agent Workflow comprising Compression (filtering noise), Perception (analyzing narrative impact on sectors), and Allocation (generating rebalancing commands). The dataset aggregates daily top-20 trending topics from four major Chinese financial platforms from 2024 to mid-2025, simulating a realistic public attention stream. Experiments evaluate nine LLMs across two tasks: macro allocation using 11 broad indices and sector rotation using 14 sector-specific ETFs. The simulation assumes a retail investor environment with 100,000 RMB initial capital, daily rebalancing at close, and a 0.01% transaction cost, comparing performance against quantitative baselines like Momentum and Mean-Variance Optimization. Results show that LLMs significantly outperform benchmarks in sector rotation during volatile regimes, generating structural alpha by identifying leading sectors. However, in low-volatility sideways markets, model performance collapses, often underperforming simple momentum strategies due to 'over-reacting to noise' and hallucinating narratives. The study highlights a 'capability trap' where larger models do not always outperform smaller ones in uncertain regimes. Limitations include the exclusion of short-selling, reliance on daily closing prices ignoring microstructure frictions, and the finding that LLMs are currently regime-dependent decision-makers rather than robust generalists.
Research problem
The paper addresses the lack of reproducible, logic-focused evaluation frameworks for financial LLM agents. Current benchmarks are either live trading platforms prone to outcome bias or static entity-centric tests that ignore the 'public attention filtering' challenge. The specific problem is evaluating an agent's ability to distill investment logic from unstructured, high-noise trending news and map it to actionable macro and sector asset allocation, rather than just predicting individual stock movements.
Core contributions
- Introduction of CN-Buzz2Portfolio, a rolling-horizon benchmark mapping daily trending news to macro and sector ETF allocation in the Chinese market.
- Development of a Tri-Stage CPA Agent Workflow (Compression, Perception, Allocation) to standardize the evaluation of LLM reasoning in financial contexts.
- Creation of a dataset simulating realistic public attention streams from four major Chinese financial platforms, avoiding pre-filtered entity news.
- Comprehensive empirical analysis of nine LLMs across two distinct market regimes (2024 volatility and 2025 oscillation), revealing regime-dependent efficacy.
- Ablation studies demonstrating the 'information utility curve' and the 'scaling law paradox' in financial decision-making.
Method
- Compression Stage: LLM summarizes noisy trending news into financially relevant events.
- Perception Stage: LLM analyzes narrative impact on specific sectors and indices.
- Allocation Stage: LLM generates rebalancing commands based on perceived opportunities and risk.
- Action Space: Structured commands for budget-based buying and ratio-based selling to minimize arithmetic errors.
- Rolling Horizon: Continuous evaluation from 2024 to 2025 to prevent memorization and test temporal generalization.
- Variance Decomposition: Statistical analysis to separate model capability from stochastic market noise.
- Ablation Studies: Testing different numbers of top news items (Top-0 to Top-20) to find optimal information density.
Results & findings
- LLMs generate significant alpha in sector rotation tasks, outperforming quantitative baselines in volatile regimes.
- In 2024, reasoning models like Qwen3-Max-Think achieved 40.11% return in sector rotation vs. 16.20% for CSI 300.
- In 2025 sideways markets, LLM performance collapsed, with some models underperforming simple momentum strategies.
- Variance decomposition shows model capability dominates stochastic noise in volatile regimes (Efficacy Ratio > 2.3) but converges to parity in stable regimes.
- Larger models do not always outperform smaller ones; Qwen3-32B outperformed Qwen3-Max in 2025 Task A, suggesting a 'capability trap'.
- Performance peaks at Top-5/Top-10 news inputs; Top-20 inputs often degrade performance due to noise and 'filter failure'.
- In 2025, 'Top-0' (price history only) often outperformed news-augmented settings, indicating news can be misleading in trendless markets.
- Memory probes show that pre-training knowledge of popular indices does not translate to accurate sector allocation logic.
Data & experiments
- Dataset: Daily top-20 trending topics from Caixin, Sina Finance, Tencent Stock, and Tiantian Fund (2024–mid-2025).
- Asset Universe: 11 macro/thematic ETFs and 14 sector-specific ETFs (e.g., New Energy, TMT, Finance).
- Simulation Environment: Retail investor setup with 100,000 RMB initial capital, daily rebalancing at market close, and 0.01% transaction costs.
- Models Evaluated: Nine LLMs including DeepSeek-R1, Qwen-3-Max, GPT-5, Gemini-2.5-Pro, and GLM-4.6.
- Baselines: Momentum strategy, Mean-Variance Optimization (MVO), CSI 300 index, and Naive Equal-Weighted Portfolio.
- Metrics: Cumulative Return, Sharpe Ratio, Maximum Drawdown, and Volatility.
Limitations & caveats
- Simulation assumes perfect liquidity and daily closing prices, ignoring slippage and market impact.
- Restricted to long-only ETFs, precluding short-selling and derivative hedging strategies.
- Operates on daily frequency, missing intraday dynamics and microstructure alpha.
- LLMs show regime-dependent efficacy, struggling in low-signal sideways markets.
- Risk of 'narrative hallucination' where models over-react to noise in trendless environments.
- Results are based on simulated retail investor constraints, limiting direct scalability to institutional portfolios.
Curator notes
Discovery status: specific_domain_focus_promoted. Source links: LLM based macro and sector asset allocation finance; CN-Buzz2Portfolio Chinese-market dataset benchmark LLM macro sector asset allocation