Temporal Data Meets LLM - Explainable Financial Time Series Forecasting

In one sentence

Summary

The paper addresses the challenge of explainable financial time series forecasting, aiming to overcome the limitations of black-box models and the difficulty of integrating multi-modal signals like news and price history. It positions Large Language Models as a unified solution capable of cross-sequence reasoning and generating interpretable outputs, focusing specifically on NASDAQ-100 stocks. The research problem centers on whether LLMs can effectively leverage textual and numerical data to predict future price movements with greater accuracy and transparency than conventional methods. The methodology involves forecasting weekly and monthly stock returns for NASDAQ-100 constituents using historical price data, company metadata, and aggregated financial news. The authors employ GPT-4 for zero-shot and few-shot inference, incorporating Chain-of-Thought (COT) prompting to enhance reasoning. Additionally, they fine-tune the open-source Open LLaMA 13B model on a dataset of 37,000 examples derived from five years of historical data. Baselines include a most-frequent heuristic, ARMA-GARCH, and a LightGBM gradient-boosting tree model with approximately 300 features. Evaluation metrics include binary precision, bin precision, Mean Squared Error (MSE) of bin ordinals, and ROUGE scores for explanation quality. Results indicate that GPT-4 with few-shot learning and COT achieves the highest performance, significantly outperforming the LightGBM baseline in both prediction accuracy and explanation quality. The fine-tuned Open LLaMA model also shows competitive performance, particularly in binary classification, though it tends to produce more extreme predictions. The study highlights that COT improves performance by enabling the model to identify crucial factors like earnings reports. Limitations include the reliance on GPT-4 for data preprocessing and the tendency of smaller models to overfit to extreme bins, suggesting that while LLMs offer explainability and strong performance, they require careful prompt engineering and potentially fine-tuning for cost-effective deployment.

Research problem

The paper tackles the difficulty of performing accurate and interpretable financial time series forecasting. Traditional models often lack explainability and struggle to integrate diverse data sources like news and price history. The specific problem is to determine if LLMs can unify these tasks by leveraging their reasoning capabilities to forecast stock returns and generate human-readable explanations based on multi-modal inputs.

Core contributions

• Proposes a novel framework for explainable financial time series forecasting using LLMs.
• Demonstrates the effectiveness of Chain-of-Thought prompting in improving LLM forecasting accuracy.
• Shows that GPT-4 outperforms traditional statistical (ARMA-GARCH) and machine learning (LightGBM) baselines.
• Validates that fine-tuned open-source models (Open LLaMA) can achieve reasonable performance with instruction tuning.
• Introduces a multi-modal input pipeline combining price data, company profiles, and aggregated news summaries.

Method

• Data preprocessing: Normalizing price changes into bins and summarizing news using GPT-4.
• Prompt engineering: Structured prompts including company profile, recent news, and few-shot examples.
• Chain-of-Thought: Adding 'reason step by step' instruction to elicit detailed reasoning.
• Fine-tuning: Instruction-based fine-tuning of Open LLaMA 13B on condensed historical data.
• Evaluation: Comparing predicted bins and explanations against ground truth using precision and ROUGE.
• Cross-sequence learning: Using examples from similar stocks to enhance few-shot performance.

Results & findings

• GPT-4 with few-shot and COT achieved 66.5% binary precision and 35.2% bin precision for weekly forecasting.
• GPT-4 outperformed the LightGBM baseline (60.8% binary precision) by approximately 5-6 percentage points.
• Chain-of-Thought prompting consistently improved performance across GPT-4 and Open LLaMA models.
• Fine-tuned Open LLaMA 13B achieved 62.2% binary precision, showing potential for open-source models.
• GPT-4 explanations scored higher on ROUGE metrics compared to zero-shot baselines.
• Open LLaMA tended to make more extreme predictions, leading to higher MSE when incorrect.
• The study confirms LLMs can effectively reason over cross-sequence information from news and prices.

Data & experiments

• Dataset: NASDAQ-100 stock price data, company metadata, and historical financial news from Yahoo Finance and Google Search API.
• Time window: 5 years for training/fine-tuning (2017-2022) and 52 weeks for evaluation (2022-2023).
• Baselines: Most-frequent historical bin, ARMA-GARCH, and LightGBM with ~300 features.
• Metrics: Binary precision, bin precision, MSE of bin ordinals, and ROUGE scores for explanation quality.
• Experiments: Zero-shot/few-shot inference with GPT-4, fine-tuning of Open LLaMA 13B, and ablation on Chain-of-Thought prompting.

Limitations & caveats

• Reliance on GPT-4 for data preprocessing may introduce bias or cost issues.
• Open LLaMA tends to make extreme predictions, increasing MSE.
• Evaluation is limited to NASDAQ-100 stocks, potentially limiting generalizability.
• News data is aggregated and summarized, which may lose granular details.
• The study does not test real-time trading performance or transaction costs.
• Fine-tuning requires significant computational resources and data preparation.

Curator notes

Discovery status: round2_promoted. Source links: A Comprehensive Review of Generative AI in Finance; Revolutionizing Finance with LLMs: An Overview of Applications and Insights; Will LLMs be Professional at Fund Investment? DeepFund: A Live Arena Perspective