Sentiment trading with large language models

In one sentence

Summary

This paper addresses the limitation of traditional dictionary-based sentiment analysis in finance by evaluating the predictive power of large language models (LLMs) on stock returns. The authors position LLMs as superior tools for extracting nuanced sentiment from unstructured text, aiming to bridge the gap between advanced NLP capabilities and practical financial forecasting. They specifically investigate whether models like OPT, BERT, and FinBERT can generate more accurate sentiment signals than the established Loughran-McDonald dictionary, which has long dominated the literature despite its simplistic approach to text representation. The methodology involves fine-tuning BERT and OPT on a dataset of 965,375 U.S. financial news articles from Refinitiv (2010-2023), using aggregated 3-day excess returns to label sentiment. The authors conduct regression analyses with firm and date fixed effects to test the correlation between model scores and next-day stock returns. They also construct daily-rebalanced, market-value-weighted long-short portfolios for each model, incorporating 10 basis points transaction costs and aligning trades with news release timings to simulate realistic trading conditions. This experimental design allows for a direct comparison of predictive accuracy and portfolio performance across different modeling approaches. Results show OPT achieving 74.4% sentiment prediction accuracy, significantly higher than BERT (72.5%), FinBERT (72.2%), and the Loughran-McDonald dictionary (50.1%). Regression coefficients confirm OPT's strong positive impact on next-day returns, while the dictionary model shows no significant relationship. The OPT-based long-short strategy yields a Sharpe ratio of 3.05 and a 355% cumulative return from 2021-2023, vastly outperforming BERT, FinBERT, and the dictionary-based strategy. Limitations include the use of open-source OPT as a proxy for GPT-4, potential overfitting in specialized models like FinBERT, and the lack of data availability for replication, though the findings strongly support the integration of advanced LLMs in quantitative finance.

Research problem

The paper investigates whether large language models (LLMs) can outperform traditional dictionary-based methods in analyzing financial news sentiment and predicting stock market returns. It addresses the gap in current literature where simplistic text mining techniques may fail to capture the nuanced context of financial news, potentially leading to suboptimal investment signals. The study specifically aims to quantify the predictive advantage of modern LLMs over established benchmarks like the Loughran-McDonald dictionary.

Core contributions

• Empirical comparison of OPT, BERT, FinBERT, and Loughran-McDonald dictionary on a large-scale dataset of U.S. financial news.
• Demonstration that OPT achieves 74.4% accuracy in sentiment prediction, significantly outperforming traditional methods.
• Development of a daily-rebalanced long-short trading strategy based on LLM sentiment scores with realistic transaction costs.
• Evidence that LLM-derived sentiment scores have robust predictive power for next-day stock returns in regression models.
• Documentation of a Sharpe ratio of 3.05 for the OPT-based strategy, highlighting the practical value of LLMs in portfolio management.

Method

• Data Collection: Refinitiv news articles matched with CRSP daily returns for U.S. stocks.
• Preprocessing: Filtering for single-stock news and removing redundant articles using cosine similarity.
• Model Fine-tuning: Supervised fine-tuning of BERT and OPT using 3-day excess returns as sentiment labels.
• Sentiment Scoring: Generating sentiment scores for each news article using the fine-tuned models.
• Regression Analysis: Linear regression with firm and date fixed effects to test return predictability.
• Portfolio Construction: Daily rebalancing of long-short portfolios based on top/bottom 20% sentiment scores.
• Transaction Costs: Applying 10 basis points per trade to simulate realistic market conditions.
• Performance Evaluation: Calculating Sharpe ratio, mean daily return, and maximum drawdown for each strategy.

Results & findings

• OPT achieves 74.4% sentiment prediction accuracy, compared to 72.5% for BERT and 50.1% for Loughran-McDonald.
• OPT regression coefficient for next-day returns is 0.274, significantly higher than BERT's 0.142 and FinBERT's 0.257.
• Loughran-McDonald dictionary shows no significant relationship with stock returns in regression analysis.
• OPT long-short strategy yields a Sharpe ratio of 3.05, outperforming BERT (2.11) and FinBERT (2.07).
• From Aug 2021 to Jul 2023, the OPT strategy generated a 355% cumulative return.
• Loughran-McDonald based strategy yielded only a 1.23 Sharpe ratio and 0.91% return in the same period.
• FinBERT underperformed general-purpose BERT, suggesting potential overfitting to financial jargon.
• Transaction costs of 10 basis points were included, ensuring realistic performance assessment.

Data & experiments

• Dataset: 965,375 U.S. financial news articles from Refinitiv (Jan 2010 - Jun 2023) matched with CRSP stock returns.
• Models: Fine-tuned BERT (345M params), OPT (2.7B params), FinBERT, and Loughran-McDonald dictionary.
• Sentiment Labeling: Supervised fine-tuning using aggregated 3-day excess returns as sentiment labels.
• Regression Analysis: Panel regressions with firm and date fixed effects to test predictive power on next-day returns.
• Portfolio Construction: Daily-rebalanced, market-value-weighted long, short, and long-short portfolios.
• Performance Metrics: Accuracy, precision, recall, F1 score, Sharpe ratio, mean daily return, and maximum drawdown.

Limitations & caveats

• OPT was used as a proxy for GPT-4, which was not publicly available, potentially limiting generalizability to the most advanced models.
• FinBERT's underperformance may indicate overfitting to financial jargon, reducing its robustness.
• The study focuses on U.S. stocks, limiting the external validity for other markets.
• Data availability is restricted, preventing independent replication of the results.
• The analysis relies on news headlines, which may not capture the full context of financial events.
• Transaction costs are simplified to a flat 10 bps, which may not reflect all market frictions.

Curator notes

Discovery status: round3_promoted. Source links: Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI; FinBERT : A Large Language Model for Extracting Information from Financial Text†; Large Language Models in equity markets: applications, techniques, and insights