Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach

In one sentence

Summary

This paper addresses the challenge of accurately predicting stock market movements following earnings announcements, a task where traditional machine learning models often fail to process extensive unstructured textual data and nuanced financial narratives. The authors argue that while historical price data is insufficient, the integration of qualitative insights from earnings reports with quantitative metrics offers a superior predictive signal. The study positions Large Language Models (LLMs) as a solution to interpret complex financial disclosures and external market conditions, aiming to provide investors with reliable, real-time forecasting tools that adapt to new information during volatile earnings seasons. The methodology involves constructing a supervised dataset of 8,556 instances from 501 S&P 500 companies, integrating 'base factors' (financial metric growth, earnings transcripts) and 'external factors' (market index performance, analyst grades, earnings surprises). Numerical and categorical data were textualized into natural language prompts. The team fine-tuned several open-source LLMs, including Llama-3-8B, Mistral-7B, and Gemma-7B, using instruction tuning and 4-bit Quantized Low-Rank Adaptation (QLoRA) for efficiency. Models were evaluated on accuracy, weighted F1, and Matthews Correlation Coefficient (MCC) against a GPT-4 baseline, using both a 'Base' dataset (internal factors only) and a 'Full' dataset (internal plus external factors). Results indicate that the fine-tuned Llama-3-8B-Instruct-4bit model achieved superior performance on the 'Full' dataset, with 16% higher accuracy and 10% better weighted F1 score compared to GPT-4. The inclusion of external factors significantly enhanced predictive power over internal factors alone. The study highlights the efficacy of QLoRA in maintaining high accuracy while reducing computational costs, making it suitable for resource-constrained deployment. Limitations include the binary 'Long/Short' output format, which ignores 'Hold' positions, and the short-term prediction horizon of only the next day, suggesting future work should expand output options and timeframes to better align with diverse investment strategies.

Research problem

The paper addresses the difficulty of accurately predicting stock price movements following earnings announcements, a period characterized by high volatility and information asymmetry. Traditional machine learning models struggle to effectively process and interpret the extensive, unstructured textual data contained in earnings reports and often overlook nuanced market signals. The research problem is to develop a robust, efficient AI-driven framework that can synthesize complex financial narratives with quantitative market data to generate superior short-term stock direction predictions compared to existing benchmarks.

Core contributions

• Proposes a novel data integration framework combining internal financial metrics and earnings transcripts with external market factors like analyst grades and index performance.
• Demonstrates the effectiveness of 4-bit QLoRA for efficient instruction fine-tuning of LLMs in financial forecasting, balancing performance with computational efficiency.
• Establishes a new benchmark for post-earnings stock prediction, showing that fine-tuned open-source LLMs can outperform proprietary models like GPT-4.
• Provides a detailed textualization pipeline for converting numerical financial data into natural language prompts suitable for LLM ingestion.
• Evaluates the relative impact of internal versus external data sources on predictive accuracy, highlighting the value of external context.

Method

• Data collection from Financial Modeling Prep API for S&P 500 companies, including transcripts, metrics, and market data.
• Textualization of numerical data (e.g., percentage changes, EPS surprises) into descriptive natural language sentences.
• Construction of instruction-following datasets with paired prompts and binary 'Long/Short' labels.
• Application of 4-bit QLoRA for parameter-efficient fine-tuning of pre-trained LLMs.
• Instruction fine-tuning using AdamW 8-bit optimizer with linear learning rate scheduling and gradient accumulation.
• Evaluation using accuracy, weighted F1, and MCC on held-out test sets for both 'Base' and 'Full' data configurations.

Results & findings

• Llama-3-8B-Instruct-4bit achieved 0.573 accuracy and 0.565 weighted F1 on the 'Full' dataset, outperforming GPT-4 (0.494 accuracy, 0.512 F1).
• The 'Full' dataset significantly improved model performance compared to the 'Base' dataset, confirming the value of external market context.
• QLoRA compression allowed for efficient fine-tuning without significant loss in predictive accuracy compared to full-precision models.
• Training loss curves showed consistent convergence, indicating stable learning dynamics during the fine-tuning process.
• Open-source models generally outperformed GPT-4 when fine-tuned with the specific financial instruction dataset.
• The study validates that LLMs can effectively process and synthesize both textual transcripts and numerical financial indicators.

Data & experiments

• Dataset of 8,556 instances covering 501 S&P 500 companies, collected via Financial Modeling Prep API.
• Features include year-over-year financial metric growth, full earnings call transcripts, market index performance (SPY, QQQ, DOW), analyst grades, and earnings surprises.
• Binary classification task: predicting next-day stock direction ('Long' if open > close, 'Short' otherwise) after earnings announcements.
• Experiments compare multiple fine-tuned open-source LLMs (Llama-3, Mistral, Gemma, Phi-3) against GPT-4 baseline.
• Evaluation metrics include accuracy, weighted F1 score, and Matthews Correlation Coefficient (MCC).
• Two dataset variants used: 'Base' (internal factors only) and 'Full' (internal plus external factors) to isolate feature impact.

Limitations & caveats

• Binary output ('Long'/'Short') ignores 'Hold' positions, which may not reflect real-world investor behavior.
• Prediction horizon is limited to the next day, potentially missing longer-term trends or strategies.
• Dataset is restricted to S&P 500 companies, limiting generalizability to small-cap or international markets.
• Reliance on textualized numerical data may lose some precision compared to direct numerical input methods.
• No backtesting on actual trading performance or transaction costs is reported, only predictive accuracy.
• Potential overfitting to specific earnings season patterns is not fully addressed.

Curator notes

Discovery status: round3_promoted. Source links: Large Language Models in equity markets: applications, techniques, and insights