The List / Trading, Investment, and Portfolio Management / paper
Ploutos: Towards interpretable stock movement prediction with financial large language model
Abstract
Recent advancements in large language models (LLMs) have opened new pathways for many domains. However, the full potential of LLMs in financial investments remains largely untapped. There are two main challenges for typical deep learning-based methods for quantitative finance. First, they struggle to fuse textual and numerical information flexibly for stock movement prediction. Second, traditional methods lack clarity and interpretability, which impedes their application in scenarios where the justification for predictions is essential. To solve the above challenges, we propose Ploutos, a novel financial LLM framework that consists of PloutosGen and PloutosGPT. The PloutosGen contains multiple primary experts that can analyze different modal data, such as text and numbers, and provide quantitative strategies from different perspectives. Then PloutosGPT combines their insights and predictions and generates interpretable rationales. To generate accurate and faithful rationales, the training strategy of PloutosGPT leverage rearview-mirror prompting mechanism to guide GPT-4 to generate rationales, and a dynamic token weighting mechanism to finetune LLM by increasing key tokens weight. Extensive experiments show our framework outperforms the state-of-the-art methods on both prediction accuracy and interpretability.
In one sentence
Summary
The paper addresses the challenge of fusing textual and numerical data for stock movement prediction while maintaining interpretability, a gap in traditional deep learning and existing LLM methods. The authors propose Ploutos, a framework comprising PloutosGen and PloutosGPT. PloutosGen utilizes a pool of specialized experts—Sentiment, Technical, and Human—to analyze diverse data modalities. The Sentiment Expert aggregates news and tweets using supervised and unsupervised fine-tuning on LLaMA-2. The Technical Expert employs a Number-to-Text Alignment (N2I-Align) technique to convert alpha formulas and time-series data into text for next-token prediction. PloutosGPT integrates these insights to generate final predictions and natural language rationales. The training of PloutosGPT introduces two key mechanisms: rearview-mirror prompting and dynamic token weighting. Rearview-mirror prompting leverages GPT-4 to generate faithful bullish and bearish rationales based on historical data and expert inputs, creating high-quality instruction data for supervised fine-tuning. Dynamic token weighting adjusts the loss function during training by emphasizing key tokens in the rationale generation process, calculated via cosine similarity between token hidden states and verbalizer type embeddings. Experiments are conducted on the ACL18 and CIKM18 datasets, which contain S&P 500 stocks with historical prices and social media text. The model is evaluated against traditional deep learning baselines (ARIMA, Adv-LSTM, StockNet) and LLM baselines (GPT-4, LLaMA-2, FinMA) using accuracy, F1, and Matthews Correlation Coefficient (MCC). Results show Ploutos-7B outperforms all baselines, achieving 61.21% accuracy and 0.205 MCC on ACL18, significantly higher than FinMA-7B (56.28% Acc) and GPT-4 (53.08% Acc). The framework also demonstrates superior interpretability, scoring 81.24% in faithfulness and 96.52% in informativeness on rationale quality, surpassing FinGPT and GPT-3. Ablation studies confirm the necessity of each expert and training component. Limitations include high computational costs, potential biases in expert selection, and the current exclusion of visual data modalities. The paper highlights the synergy between predictive accuracy and interpretability, suggesting that LLMs can serve as effective decision-support tools in quantitative finance when properly trained with multimodal and explainable constraints.
Research problem
The paper tackles the difficulty of effectively fusing multimodal data (textual news and numerical time-series) for stock movement prediction while ensuring the model's decisions are interpretable. Traditional deep learning methods lack clarity, and existing LLM approaches often fail to generate faithful rationales or adaptively combine diverse financial signals.
Core contributions
- Proposes PloutosGen, a pipeline with diverse experts (Sentiment, Technical, Human) to analyze multimodal stock data.
- Introduces N2I-Align, a method to align numerical alpha factors with text for LLM-based technical analysis.
- Develops PloutosGPT, an LLM framework for interpretable stock prediction using rearview-mirror prompting.
- Implements dynamic token weighting to enhance the accuracy and faithfulness of generated rationales during fine-tuning.
- Establishes new metrics for evaluating the faithfulness and informativeness of financial LLM rationales.
Method
- PloutosGen uses separate LLM experts for sentiment analysis (fine-tuned on FIQA-SA, NWGI, etc.) and technical analysis.
- Technical expert uses N2I-Align to convert alpha formulas and time-series values into text sequences for next-token prediction.
- Rearview-mirror prompting uses GPT-4 to generate historical rationales for known price movements to create training data.
- Dynamic token weighting modifies the cross-entropy loss by weighting tokens based on cosine similarity with verbalizer embeddings.
- PloutosGPT is fine-tuned on LLaMA-2 using the curated instruction data from rearview-mirror prompting.
- Final prediction is made by aggregating expert inputs and generating a natural language rationale with a binary up/down label.
Results & findings
- Ploutos-7B achieved 61.21% accuracy and 0.205 MCC on ACL18, outperforming FinMA-7B (56.28% Acc) and GPT-4 (53.08% Acc).
- On CIKM18, Ploutos-7B reached 59.89% accuracy and 0.064 MCC, surpassing other LLM baselines.
- Ploutos demonstrated superior faithfulness (81.24%) and informativeness (96.52%) in rationale generation compared to FinGPT and GPT-3.
- Ablation studies showed that removing any expert or training component significantly degraded performance.
- Dynamic token weighting with a temperature of 0.5 optimized the trade-off between prediction accuracy and rationale faithfulness.
- Zero-shot LLaMA-2 and GPT-4 performed near random guessing, highlighting the need for domain-specific fine-tuning.
Data & experiments
- Evaluated on ACL18 and CIKM18 datasets containing S&P 500 stocks, historical prices, and social media text.
- Compared against traditional baselines (ARIMA, Adv-LSTM, StockNet, DTML) and LLM baselines (GPT-4, LLaMA-2, FinMA).
- Used accuracy, F1 score, and Matthews Correlation Coefficient (MCC) for prediction performance evaluation.
- Assessed rationale quality using faithfulness (inference from given knowledge) and informativeness metrics.
- Conducted ablation studies to isolate the impact of individual experts and training mechanisms.
Limitations & caveats
- High computational cost associated with training multiple experts and using GPT-4 for data generation.
- Performance is sensitive to the selection and quality of the primary experts.
- Current model does not incorporate visual data, limiting its multimodal scope.
- Potential biases in expert predictions may propagate to the final PloutosGPT output.
- Evaluation is limited to specific datasets (ACL18, CIKM18) and may not generalize to all market conditions.
Curator notes
Discovery status: round3_promoted. Source links: Exploring the Synergy of Quantitative Factors and Newsflow Representations from Large Language Models for Stock Return Prediction; Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning