LLMFactor: Extracting Profitable Factors through Prompts for Explainable Stock Movement Prediction

In one sentence

Summary

The paper addresses the challenge of explainable stock movement prediction by proposing LLMFactor, a framework that extracts specific market factors from textual news rather than relying on generic sentiment or keyphrases. The authors argue that factors provide better human readability and direct correlation to price dynamics, addressing the black-box nature of traditional deep learning models and the limited interpretability of sentiment analysis. The research positions this approach as a novel task in financial NLP, aiming to bridge the gap between unstructured text analysis and structured time-series forecasting. The core method, Sequential Knowledge-Guided Prompting (SKGP), operates in three steps: first, it uses a fill-in-the-blank strategy to identify relationships between the target stock and related companies in the news; second, it extracts the top-k factors influencing the stock price; and third, it combines these factors with historical price movements converted into text to predict future direction. The study evaluates this framework using GPT-3.5-turbo, GPT-4, and GPT-4-turbo on four benchmark datasets (StockNet, CMIN-US, CMIN-CN, and EDT), comparing performance against keyphrase-based, sentiment-based, and time-series baselines using Accuracy and Matthews Correlation Coefficient (MCC). Results show LLMFactor outperforms state-of-the-art methods, with GPT-4 achieving the highest MCC scores across US datasets. Ablation studies reveal that the extracted factors contribute significantly to performance gains, particularly in MCC, while historical price data remains the strongest predictor. The framework demonstrates strong explainability through case studies on Apple and Tesla. However, limitations include reliance on LLM API variability, reduced performance on Chinese datasets due to language proficiency gaps in GPT models, and the challenge of converting time-series data into text without losing nuance.

Research problem

The paper addresses the difficulty of predicting stock movements using Large Language Models (LLMs) due to their lack of native time-series handling and the poor explainability of existing text-based methods like sentiment analysis or keyphrase extraction. It seeks to introduce a framework that extracts specific, interpretable 'factors' from news to improve both prediction accuracy and the ability to explain temporal price changes to humans.

Core contributions

• Introduction of a novel 'factor extraction' task that surpasses keyphrase and sentiment methods in explainability and stock price correlation.
• Proposal of Sequential Knowledge-Guided Prompting (SKGP), a three-step prompting strategy using fill-in-the-blank techniques to generate background knowledge and factors.
• Development of the LLMFactor framework, which integrates extracted factors, company relations, and textualized historical prices for binary stock movement prediction.
• Comprehensive evaluation across four benchmark datasets (US and Chinese markets) demonstrating superiority over keyphrase, sentiment, and time-series baselines.

Method

• Step 1: Match target stock with related stocks in news and prompt LLM to fill in the blank for their relationship (e.g., competitor).
• Step 2: Prompt LLM to extract top-k factors from the news text that may affect the stock price.
• Step 3: Convert historical price movements (rise/fall) into text sequences.
• Step 4: Construct a final prompt combining relations, factors, and historical price text to predict the next day's movement (rise/fall) and provide reasoning.
• Used GPT-3.5-turbo, GPT-4, and GPT-4-turbo via API for inference.
• Window size of 5 days for historical price context.

Results & findings

• LLMFactor with GPT-4-turbo achieved an MCC of 0.228 on StockNet, outperforming the best baseline (CMIN at 0.209).
• On CMIN-US, LLMFactor with GPT-4 achieved an MCC of 0.284, surpassing GPT-4-turbo sentiment baselines and time-series models.
• Ablation studies show the 'factor' layer contributes 46% of the MCC improvement over using only price data.
• Performance on CMIN-CN (Chinese market) was lower (MCC ~0.19) compared to US datasets, indicating language bias in GPT models.
• Keyphrase-based methods showed negligible performance gains (MCC near 0), validating the need for factor extraction.
• The EDT dataset, lacking historical price data, yielded lower accuracy (~59%), highlighting the importance of temporal context.

Data & experiments

• Evaluation on four datasets: StockNet (US, tweets/price), CMIN-US (US, tweets/price), CMIN-CN (China, tweets/price), and EDT (US, news/price).
• Comparison against keyphrase baselines (PromptRank, KeyBERT, TextRank), sentiment baselines (FinBERT, RoBERTa, GPT variants), and time-series baselines (CMIN, StockNet).
• Metrics used are Accuracy (ACC) and Matthews Correlation Coefficient (MCC).
• Ablation study isolating the contribution of price data, factors, and company relations to overall performance.
• Case studies on Apple (AAPL) and Tesla (TSLA) to demonstrate interpretability.

Limitations & caveats

• Reliance on LLM API responses introduces variability and reproducibility challenges.
• Performance drops on Chinese datasets, suggesting language bias in current GPT models.
• Converting time-series data to text may lose numerical precision or nuance.
• Quality of factors depends on the LLM's ability to interpret news, which may vary by text length and type.
• Evaluation is limited to binary classification (rise/fall), not continuous price prediction.

Curator notes

Discovery status: round4_promoted. Source links: Combining Financial Data and News Articles for Stock Price Movement Prediction Using Large Language Models; Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading