ChatGPT Informed Graph Neural Network for Stock Movement Prediction

In one sentence

Summary

The paper addresses the challenge of capturing latent inter-dependencies among equities for stock movement prediction, arguing that existing methods often fail to model dynamic relationships and the lead-lag effects caused by news propagation. It positions Large Language Models (LLMs) as superior tools for extracting these relationships from unstructured text compared to static knowledge graphs or handcrafted features. The research aims to leverage ChatGPT's zero-shot inference capabilities to build evolving network structures from daily financial news, thereby enhancing the predictive power of downstream deep learning models. The proposed method integrates three components: network structure inference, graph embedding, and sequential prediction. First, ChatGPT is prompted to identify target companies affected by daily news headlines and their sentiment, constructing a dynamic graph where edges connect companies mentioned together. Second, a Graph Neural Network (GNN) generates node embeddings by aggregating information from these inferred neighbors. Third, these embeddings are concatenated with historical stock market data and processed by Long Short-Term Memory (LSTM) networks to classify next-day stock movements as up, down, or neutral. The model is evaluated on the DOW 30 companies using data from September 2020 to December 2022, with a test period starting October 2021 to avoid data leakage from ChatGPT's training cutoff. Experimental results show the model consistently outperforms baselines (LSTM, News-Embed, ChatGPT sentiment-only) in weighted, Micro, and Macro F1 scores, with a minimum improvement of 1.8%. Portfolio backtesting reveals that strategies based on the model's predictions yield higher annualized cumulative returns, lower volatility, and reduced maximum drawdown compared to benchmarks. The study highlights that while ChatGPT sentiment alone struggles with downtrend prediction, the graph-informed approach improves detection of both upward and downward movements. Limitations include the small sample size (DOW 30), potential oversmoothing in GNNs, and reliance on basic network structures without sentiment edge attributes.

Research problem

The paper addresses the difficulty of accurately predicting stock price movements by capturing latent, dynamic inter-dependencies among equities that traditional models miss. It specifically targets the challenge of identifying which companies are affected by news and how information propagates through the market (lead-lag effects). The problem is framed as a need for a more adaptive and comprehensive method to construct stock networks from unstructured textual data compared to static knowledge graphs.

Core contributions

• First study to use ChatGPT for inferring dynamic network structures from temporal financial news.
• Novel framework combining LLM-based graph inference with Graph Neural Networks (GNN) for stock prediction.
• Demonstration that ChatGPT-inferred networks improve stock movement classification over static baselines.
• Empirical evidence of superior portfolio performance (returns, volatility, drawdown) using model outputs.
• Proposal of a zero-shot learning approach for constructing financial graphs without manual feature engineering.

Method

• Prompt engineering: ChatGPT identifies affected companies and sentiment from news headlines.
• Graph construction: Edges created between companies mentioned together in ChatGPT's output.
• GNN Embedding: Aggregates neighbor information to create node representations.
• Feature Concatenation: Combines GNN embeddings with historical stock market data.
• Sequential Modeling: Uses LSTM to process time-series sequences of combined features.
• Classification: Fully connected layer predicts up/down/neutral movement using cross-entropy loss.

Results & findings

• Model outperforms all baselines in F1 metrics with a minimum 1.8% improvement.
• Long-only portfolio based on model predictions shows higher cumulative returns than benchmarks.
• Long-short portfolio exhibits lower annualized volatility (14.06% vs 23.61% for ChatGPT baseline).
• Maximum drawdown is significantly reduced (0.1242 vs 0.2112 for ChatGPT baseline).
• ChatGPT sentiment-only model performs poorly on downtrend prediction (10.88% F1) compared to the proposed model (19.46%).
• Marginal improvements in F1 score translate to significant potential profit increases in trading contexts.

Data & experiments

• Dataset: DOW 30 companies from September 2020 to December 2022.
• News Data: Over 115,000 unique headlines from reputable media outlets mentioning DOW 30 stocks.
• Market Data: Daily OHLCV and dividend data from CRSP databases.
• Benchmarks: LSTM, ARIMA, News-Embed MLP, and ChatGPT sentiment-only model.
• Metrics: Weighted F1, Micro F1, Macro F1 for classification; Cumulative Return, Volatility, Sharpe Ratio, Max Drawdown for portfolios.
• Test Period: October 2021 to December 2022 to ensure out-of-sample validity relative to ChatGPT's training data.

Limitations & caveats

• Limited to DOW 30 companies, which may not generalize to broader markets.
• Basic network structures used; more sophisticated GNN architectures could be explored.
• Reliance on ChatGPT's zero-shot capabilities without fine-tuning.
• Potential for data leakage if news timestamps are not carefully aligned with market close.
• Does not incorporate sentiment scores as edge attributes in the graph.
• ChatGPT's training data cutoff limits its ability to process post-2021 news without browsing capabilities.

Curator notes

Discovery status: round3_promoted. Source links: Assessing Look-Ahead Bias in Stock Return Predictions Generated by GPT Sentiment Analysis; Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI