Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI

In one sentence

Summary

This paper addresses the challenge of effective stock selection in complex markets by introducing MarketSenseAI, a framework that emulates expert investment decision-making using GPT-4. The system integrates Chain of Thought and In-Context Learning to process diverse data sources, including market trends, news, fundamentals, and macroeconomic factors. By generating actionable and interpretable investment signals, the framework aims to mitigate human cognitive biases and enhance the quality of financial analysis for both retail and professional investors. The approach bridges domain knowledge with advanced AI reasoning to provide holistic stock insights. The methodology employs a modular architecture with five core components: a Progressive News Summarizer, a Fundamentals Summarizer, a Stock Price Dynamics Summarizer, a Macroeconomic Summarizer, and a final recommendation engine. Data is sourced from EODHD APIs for news and fundamentals, while peer stocks are identified using MPNet embeddings. The system was empirically tested on S&P 100 stocks over a 15-month period. GPT-4 was used not only for prediction but also as a signal evaluator to rank the quality of explanations. Performance was measured against benchmarks like the S&P 500 and various naive strategies, accounting for transaction costs. Results demonstrate exceptional performance, with the top strategy achieving a cumulative return of 72.87% (71.64% after costs) and an excess alpha of 10-30%, while maintaining a risk profile comparable to the broader market. The study highlights the value of AI-generated explanations in improving signal reliability and acceptance. However, limitations include the short evaluation period, potential overfitting to specific market conditions, and the commercial affiliation of the authors. The findings suggest significant potential for LLMs in financial analytics but call for extended research across diverse market environments.

Research problem

The paper addresses the difficulty of effective stock selection in modern financial markets, where price discovery is disrupted by passive investing, retail behavior, and complex macroeconomic factors. It aims to leverage Large Language Models to emulate expert investment decision-making, providing actionable and interpretable signals that mitigate human cognitive biases and handle diverse data sources more efficiently than traditional methods.

Core contributions

• Introduction of MarketSenseAI, an LLM-driven framework for holistic stock analysis.
• Integration of Chain of Thought and In-Context Learning for enhanced reasoning.
• Development of modular components for news, fundamentals, and price dynamics.
• Use of GPT-4 as both a predictive engine and a signal evaluator for explanation quality.
• Empirical validation on S&P 100 stocks showing superior performance over benchmarks.
• Demonstration of explainable investment signals to improve transparency and trust.

Method

• Progressive News Summarizer aggregates and updates daily news summaries monthly.
• Fundamentals Summarizer standardizes and compares quarterly financial statements.
• Stock Price Dynamics Summarizer compares target stock with peers and S&P 500.
• MPNet embeddings used to identify similar peer stocks for comparative analysis.
• GPT-4 employed with Chain of Thought and In-Context Learning for reasoning.
• GPT-4 acts as a secondary evaluator to rank the quality of investment signals.
• Final recommendation engine synthesizes all summaries into buy/hold/sell signals.

Results & findings

• MarketSenseAI achieved a cumulative return of up to 72.87% over the evaluation period.
• Excess alpha generated ranged from 10% to 30% compared to market benchmarks.
• The MS-Top10-Cap-GPT strategy outperformed all other evaluated strategies.
• Risk profile remained comparable to the broader market despite high returns.
• GPT-4 ranking of explanations correlated with higher signal quality and acceptance.
• Technology and AI-related stocks received higher scores, reflecting market sentiment.
• Performance held up after accounting for transaction costs of 5bps per trade.

Data & experiments

• Evaluated on S&P 100 stocks over a 15-month period.
• Data sourced from EODHD APIs for news and fundamental financial statements.
• Peer stocks identified using MPNet embeddings for comparative analysis.
• Benchmarks include S&P 500 index and various naive portfolio strategies.
• Performance metrics include cumulative return, Sharpe ratio, Sortino ratio, and max drawdown.
• Transaction costs of 5bps per trade were included in performance calculations.

Limitations & caveats

• Evaluation period of 15 months may not capture long-term market cycles.
• Potential overfitting to specific market conditions during the test period.
• Authors have commercial affiliation with the MarketSenseAI product.
• Limited generalizability to markets outside the S&P 100 or US equities.
• Reliance on GPT-4 API may incur significant costs for large-scale deployment.
• Does not account for all behavioral biases inherent in LLM training data.

Curator notes

Discovery status: round2_promoted. Source links: Advancing Financial Engineering with Foundation Models: Progress, Applications, and Challenges; Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks; FinBERT: Financial Sentiment Analysis with Pre-trained Language Models