MarketSenseAI 2.0: Enhancing Stock Analysis Through LLM Agents

In one sentence

Summary

MarketSenseAI 2.0 addresses the fragmentation in stock analysis by integrating heterogeneous data sources—financial news, historical prices, company fundamentals, and macroeconomic environments—into a unified decision-making framework. The system aims to overcome limitations of traditional quantitative models that often ignore textual context and qualitative insights, providing transparent, explainable investment signals through a Chain-of-Agents (CoA) architecture. By combining Retrieval-Augmented Generation (RAG) with specialized LLM agents, the framework processes complex documents like 10-K filings and earnings call transcripts, enriching macroeconomic analysis with institutional reports to support holistic stock selection. The methodology employs five distinct LLM agents: News, Fundamentals, Dynamics, Macroeconomic, and Signal. The Fundamentals Agent uses a three-layer approach to summarize SEC filings and earnings calls, integrating qualitative managerial tone with quantitative financial ratios. The Macroeconomic Agent utilizes semantic chunking and Hypothetical Dense Embeddings (HyDE) for retrieval from a vector datastore containing central bank and investment bank reports. Empirical evaluations were conducted on S&P 100 stocks over 2023-2024 and S&P 500 stocks in 2024, using VectorBTPro for backtesting and Ragas for RAG evaluation. The system generates monthly long-only portfolios based on buy signals, comparing performance against benchmark indices and factor models. Results indicate significant outperformance, with MarketSenseAI achieving 125.9% cumulative returns on the S&P 100 versus 73.5% for the index, and a 33.8% higher Sortino ratio on the S&P 500. Factor analysis reveals unexplained alpha of 8.0%, suggesting the model captures idiosyncratic opportunities beyond traditional value and momentum factors. The inclusion of textual data from filings and calls moderates sentiment scores, revealing hidden risks and shifting approximately 5% of investment signals. Limitations include reliance on GPT-4o, potential context window constraints, and the need for robust prompt engineering to ensure reproducibility and mitigate hallucinations in financial reasoning.

Research problem

The paper addresses the challenge of holistic stock analysis by integrating diverse, heterogeneous data sources including financial news, historical prices, company fundamentals, and macroeconomic environments. It aims to overcome the limitations of existing systematic approaches that often rely on isolated data types or black-box models, lacking transparency and comprehensive qualitative context. The framework seeks to provide explainable, data-driven investment signals that leverage the full scope of available financial information.

Core contributions

• Introduction of a Chain-of-Agents (CoA) architecture for granular handling of large-scale financial data like 10-Q, 10-K reports, and earnings call transcripts.
• Development of an enhanced RAG module for macroeconomic analysis using semantic chunking and HyDE-based retrieval to process diverse institutional reports.
• Demonstration of significant improvement in fundamental analysis accuracy by integrating qualitative insights from SEC filings and earnings calls with quantitative metrics.
• Empirical validation on S&P 100 (2023-2024) and S&P 500 (2024) stocks showing cumulative returns of 125.9% and a 33.8% higher Sortino ratio compared to benchmarks.
• Provision of detailed explanations for investment decisions, enhancing transparency and user trust compared to traditional black-box quantitative models.

Method

• Chain-of-Agents (CoA) architecture with five specialized LLM agents: News, Fundamentals, Dynamics, Macroeconomic, and Signal.
• Fundamentals Agent uses a three-layer process: Filing Summary, Earnings Call Summary, and Fundamental Consolidation.
• Macroeconomic Agent employs data injection scripts, LLM-powered classification, and semantic chunking for vector storage.
• Retrieval-Augmented Generation (RAG) with HyDE for macroeconomic queries and optimized retrieval for open-ended queries.
• Sentiment analysis using FinBERT to evaluate the impact of textual data on agent outputs.
• Portfolio construction and backtesting using VectorBTPro with monthly rebalancing and transaction cost consideration.
• Factor model decomposition using Carhart 4-Factor and Fama-French 5-Factor models.

Results & findings

• MarketSenseAI achieved cumulative returns of 125.9% on S&P 100 stocks (2023-2024) compared to the index return of 73.5%.
• On S&P 500 stocks in 2024, the framework delivered a 33.8% higher Sortino ratio than the market.
• Incorporating SEC filings and earnings calls moderated sentiment scores, revealing hidden risks and shifting ~5% of investment signals.
• Factor analysis showed insignificant loadings on profitability (RMW) and investment (CMA), suggesting returns are not driven by these traditional factors.
• The model generated a significant residual alpha of +8.0%, with 12-15% of returns unexplained by traditional risk factors.
• HyDE retrieval method outperformed simple and optimized methods in context recall and answer relevancy for complex macroeconomic queries.

Data & experiments

• Evaluated on S&P 100 stocks for a two-year period (2023-2024) and S&P 500 stocks for 2024.
• Input data includes financial news, quarterly statements, SEC filings, earnings call transcripts, historical price data, and macroeconomic reports from central banks and investment banks.
• Backtesting performed using VectorBTPro to account for transaction costs and generate monthly long-only portfolios.
• RAG retrieval performance evaluated using Ragas framework with metrics like context recall, precision, answer relevancy, and faithfulness.
• Factor analysis conducted using Carhart 4-Factor and Fama-French 5-Factor models to decompose returns and identify alpha sources.

Limitations & caveats

• Reliance on GPT-4o may limit generalizability to other LLM architectures.
• Context window constraints of LLMs can still pose challenges for processing extremely large documents.
• Model outputs are sensitive to prompt engineering, affecting reproducibility and backtesting consistency.
• Potential for hallucinations in financial reasoning requires careful governance and validation.
• Evaluation limited to US large-cap stocks (S&P 100/500), limiting external validity for other markets.
• Macroeconomic analysis relies on scraped reports, which may have coverage or timeliness biases.

Curator notes

Discovery status: round3_promoted_seed_for_round4. Source links: Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI; FinBERT : A Large Language Model for Extracting Information from Financial Text†