The List / Trading, Investment, and Portfolio Management / paper
ECC Analyzer: Extracting Trading Signal from Earnings Conference Calls using Large Language Model for Stock Volatility Prediction
Abstract
In the realm of financial analytics, leveraging unstructured data, such as earnings conference calls (ECCs), to forecast stock volatility is a critical challenge that has attracted both academics and investors. While previous studies have used multimodal deep learning-based models to obtain a general view of ECCs for volatility predicting, they often fail to capture detailed, complex information. Our research introduces a novel framework: ECC Analyzer, which utilizes large language models (LLMs) to extract richer, more predictive content from ECCs to aid the model’s prediction performance. We use the pre-trained large models to extract textual and audio features from ECCs and implement a hierarchical information extraction strategy to extract more fine-grained information. This strategy first extracts paragraph-level general information by summarizing the text and then extracts fine-grained focus sentences using Retrieval-Augmented Generation (RAG). These features are then fused through multimodal feature fusion to perform volatility prediction. Experimental results demonstrate that our model outperforms traditional analytical benchmarks, confirming the effectiveness of advanced LLM techniques in financial analysis.
In one sentence
Summary
The paper addresses the challenge of forecasting stock volatility by extracting rich, predictive signals from unstructured earnings conference call (ECC) data. While prior multimodal approaches often treat text and audio as flat inputs, missing nuanced context, this work positions LLMs as essential tools for hierarchical information extraction. The core problem is to move beyond general sentiment analysis to capture specific, high-impact financial details that drive market reactions, thereby improving the accuracy of volatility forecasts for S&P 500 stocks. The proposed ECC Analyzer framework employs a multi-stage pipeline. It extracts audio embeddings using Wav2Vec2 and text embeddings using SimCSE, processed through Multi-Head Self-Attention. Crucially, it implements a hierarchical extraction strategy: LLMs summarize ECC transcripts at paragraph and chunk levels, while a Retrieval-Augmented Generation (RAG) system uses a financial expert-designed 'Question Bank' to retrieve fine-grained focus sentences. These diverse features—raw audio, raw text, summaries, and RAG-extracted sentences—are fused via additive interactions and fed into a regression model. Experiments use the S&P 500 ECC dataset, comparing against GARCH, LSTM, and multimodal baselines like HTML and MRDM, evaluating performance using Mean Squared Error (MSE) across 3, 7, 15, and 30-day horizons. Results show ECC Analyzer outperforms all baselines, particularly in short-term (3-day and 7-day) volatility prediction, reducing average MSE by 27.7% compared to the best existing model. Ablation studies confirm that fine-grained RAG-extracted sentences contribute most significantly to performance gains, whereas raw LLM direct prediction is ineffective. The study highlights that LLMs serve best as feature extractors rather than direct predictors. Limitations include reliance on specific financial questions for RAG and potential overfitting to short-term signals, with medium-term performance comparable but not superior to specialized baselines like AMA-LSTM.
Research problem
The paper tackles the difficulty of accurately predicting stock volatility using unstructured earnings conference call (ECC) data. Existing multimodal models often fail to capture detailed, complex information by treating text and audio features uniformly. The research aims to leverage Large Language Models (LLMs) to extract richer, hierarchical, and fine-grained signals from ECCs to enhance prediction accuracy, addressing the gap where previous methods miss critical contextual nuances.
Core contributions
- Introduction of ECC Analyzer, a novel framework integrating multimodal ECC data with LLM-based hierarchical information extraction.
- Development of a fine-grained extraction strategy using Retrieval-Augmented Generation (RAG) guided by a financial expert-designed 'Question Bank'.
- Demonstration of a 27.7% reduction in average Mean Squared Error (MSE) compared to state-of-the-art baselines, especially for short-term volatility.
- Empirical evidence that LLMs are effective as feature extractors for financial analysis but ineffective as direct volatility predictors.
- Comprehensive ablation study isolating the contribution of audio, text, summaries, and RAG-extracted sentences to prediction performance.
Method
- Audio Encoding: Wav2Vec2 for audio embeddings, followed by Multi-Head Self-Attention and average pooling.
- Text Encoding: SimCSE for sentence embeddings, processed through Multi-Head Self-Attention and pooling.
- Hierarchical Summarization: LLMs summarize ECC chunks and then aggregate them into an overall summary.
- RAG Extraction: A 'Question Bank' queries the ECC text to retrieve and generate fine-grained focus sentences.
- Feature Fusion: Additive interaction layer combines audio, text, summary, and RAG features into a unified representation.
- Prediction: Fully connected layers perform regression on the fused features to predict volatility.
- Optimization: Model trained using Adam optimizer with MSE loss function.
Results & findings
- ECC Analyzer achieves the lowest MSE for 3-day (0.553) and 7-day (0.306) volatility prediction among all tested models.
- The framework reduces average MSE by 27.7% compared to the current state-of-the-art model (HTML).
- Direct prediction using GPT-4 Turbo yields poor results (MSE 2.198), indicating LLMs are not reliable direct predictors.
- Ablation studies show that RAG-extracted fine-grained sentences provide the most significant performance boost.
- Long-term (30-day) performance is comparable to SOTA methods, while medium-term (15-day) is slightly lower than AMA-LSTM.
- Multimodal fusion of audio and text features significantly improves short-term accuracy over unimodal approaches.
Data & experiments
- Dataset: Publicly available S&P 500 ECC dataset containing audio and transcripts from 2017 earnings calls of 500 companies.
- Split: Temporal 8:2 train-test split to ensure historical data predicts future volatility.
- Baselines: GARCH, LSTM, MT-LSTM-ATT, HAN (Glove), MRDM, HTML, AMA-LSTM, and direct GPT-4 Turbo prediction.
- Metrics: Mean Squared Error (MSE) for 3, 7, 15, and 30-day volatility horizons.
- Ablation Study: Systematic removal of components (audio, text, summaries, RAG sentences) to assess individual contributions.
Limitations & caveats
- The model's performance on medium-term (15-day) volatility is slightly inferior to specialized baselines like AMA-LSTM.
- Reliance on a fixed 'Question Bank' may limit the model's ability to capture novel or unexpected financial signals not pre-defined by experts.
- Direct LLM prediction is shown to be ineffective, suggesting high costs and latency for real-time trading applications if not optimized.
- The study focuses on S&P 500 stocks, potentially limiting generalizability to smaller caps or other markets.
- The temporal split uses 2017 data, which may not reflect current market dynamics or LLM capabilities in later years.
Curator notes
Discovery status: trading_agent_focus_promoted. Source links: FinMem: A Performance-Enhanced Large Language Model Trading Agent with Layered Memory and Character Design