FinBERT-FOMC: Fine-Tuned FinBERT Model with Sentiment Focus Method for Enhancing Sentiment Analysis of FOMC Minutes

In one sentence

Summary

This research addresses the challenge of accurately analyzing sentiment in complex financial texts, specifically Federal Open Market Committee (FOMC) Minutes, where standard models struggle with nuanced, contradictory sentences. The authors identify that FinBERT, while effective for simple financial statements, misclassifies sentences containing disjunctive conjunctions like 'but', 'while', and 'though' because it fails to isolate the primary sentiment-bearing clause. The study aims to enhance predictive performance by combining a novel text simplification strategy with domain-specific fine-tuning, targeting the specific linguistic complexities inherent in central bank communications. The methodology involves curating a dataset of 32,330 sentences from FOMC Minutes (2006-2023) and manually labeling a test set of 1,375 entries. The core innovation is the Sentiment Focus (SF) method, a preprocessing step that removes clauses preceding contrasting conjunctions to simplify sentence structure before classification. The authors fine-tuned FinBERT on a subset of 3,535 complex sentences processed with SF. Experiments compared the original FinBERT, FinBERT with SF preprocessing, and the fine-tuned FinBERT-FOMC model using accuracy as the primary metric on the manually labeled test set. Results show that the fine-tuned FinBERT-FOMC model achieved an overall accuracy of 88.29%, a 5% improvement over the original FinBERT's 84.07%. The improvement was most pronounced for complex sentences, where the fine-tuned model outperformed the original by 17.4% (87.1% vs. 74.2% accuracy). Specifically, accuracy on sentences with 'but' rose from 67.3% to 86.1%, and on 'though' from 73.1% to 90.4%. The study concludes that simplifying complex syntax via SF and fine-tuning on domain-specific complex examples significantly enhances sentiment analysis robustness, though limitations include the small manual test set and lack of generalization testing on other financial document types.

Research problem

Standard financial NLP models, particularly FinBERT, struggle to accurately classify sentiment in complex FOMC Minutes sentences that contain contrasting conjunctions (e.g., 'but', 'while', 'though'). These sentences often mask the primary sentiment in subordinate clauses, leading to misclassification by models that do not isolate the main sentiment-bearing clause. The research problem is to improve the predictive accuracy of sentiment analysis for these specific, linguistically complex financial texts.

Core contributions

• Identification of a specific failure mode in FinBERT regarding complex sentences with contrasting conjunctions in FOMC Minutes.
• Proposal of the Sentiment Focus (SF) method, a preprocessing technique that simplifies sentence structure by removing non-sentiment-bearing clauses before conjunctions.
• Development of FinBERT-FOMC, a fine-tuned model trained on SF-processed complex sentences to better capture nuanced financial sentiment.
• Creation of a manually labeled test dataset of 1,375 FOMC Minute sentences for rigorous evaluation of sentiment classification performance.
• Demonstration of significant accuracy improvements (17.4% on complex sentences) over baseline FinBERT using the proposed method.

Method

• Data Extraction: Retrieved FOMC Minutes from the Federal Reserve website (2006-2023) and segmented into sentences.
• Sentiment Focus (SF) Preprocessing: Identified sentences with contrasting conjunctions ('but', 'while', 'though', 'although') and removed preceding clauses to isolate the main sentiment-bearing segment.
• Complex Sentence Identification: Filtered the dataset to isolate 3,535 sentences modified by SF, defining the 'complex' subset.
• Fine-Tuning: Trained FinBERT on the SF-processed complex sentences for 5 epochs with a learning rate of 2e-5.
• Evaluation: Compared original FinBERT, SF-enhanced FinBERT, and Fine-tuned FinBERT-FOMC on a manually labeled test set.
• Error Analysis: Analyzed misclassifications to identify patterns related to conjunctions and sentence complexity.

Results & findings

• Original FinBERT achieved 84.07% accuracy on the manually labeled test set.
• FinBERT with Sentiment Focus preprocessing improved accuracy to 85.75%.
• Fine-tuned FinBERT-FOMC achieved 88.29% accuracy, a 5.02% overall improvement.
• On complex sentences only, FinBERT-FOMC reached 87.10% accuracy vs. 74.19% for original FinBERT (17.4% relative improvement).
• For sentences with 'but', accuracy improved from 67.33% to 86.14%.
• For sentences with 'while', accuracy improved from 87.50% to 100.00%.
• For sentences with 'though', accuracy improved from 73.08% to 90.38%.
• The SF method successfully simplified 3,535 out of 32,330 sentences, indicating a significant portion of FOMC text is complex.

Data & experiments

• Dataset: 32,330 sentences extracted from FOMC Minutes (January 2006 to February 2023).
• Test Set: Manually labeled subset of 1,375 sentences, categorized as positive, negative, or neutral by human experts.
• Complex Sentence Subset: 3,535 sentences identified as complex (modified by SF preprocessing) used for fine-tuning.
• Baselines: Original FinBERT, FinBERT with Sentiment Focus preprocessing, and Fine-tuned FinBERT-FOMC.
• Metrics: Accuracy for three-class classification (positive, negative, neutral).
• Evaluation: Comparison of model performance on the full test set and specifically on the complex sentence subset.

Limitations & caveats

• The manually labeled test set is small (1,375 entries), potentially limiting statistical power and generalizability.
• Manual labeling introduces potential human bias and error, despite expert judgment.
• The study only evaluates FinBERT; other advanced models (e.g., BloombergGPT, GPT-4) were not directly tested in the main experiment.
• The Sentiment Focus method is specific to sentences with contrasting conjunctions and may not generalize to all financial text complexities.
• The dataset is limited to FOMC Minutes, so results may not apply to other financial documents with different linguistic structures.
• The study does not address other financial NLP challenges like jargon or entity recognition, focusing solely on sentiment.

Curator notes

Discovery status: round2_promoted. Source links: FinBERT: A Pretrained Language Model for Financial Communications; FinBERT: Financial Sentiment Analysis with Pre-trained Language Models; FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications