Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucination

In one sentence

Summary

The paper addresses the critical issue of hallucination in Large Language Models (LLMs) when applied to the finance domain, where factual accuracy is paramount. It positions the study as an empirical examination of FinLLM reliability, highlighting that while LLMs show promise, their tendency to generate plausible but incorrect information poses significant risks for monetary loss and trust erosion. The authors argue that existing research lacks sufficient empirical investigation into the specific nature and extent of these hallucinations in financial contexts, necessitating a rigorous benchmark to assess model capabilities and failure modes. The methodology involves evaluating several models, including Llama2, GPT-3.5, GPT-4, and the domain-specific FinMA, across three distinct financial tasks: financial abbreviation recognition, financial term explanation, and historical stock price querying. The authors employ metrics such as accuracy, FactScore for factual consistency, and Mean Absolute Error (MAE) for price predictions. To mitigate hallucinations, they test four strategies: few-shot learning, Decoding by Contrasting Layers (DoLa), Retrieval Augmented Generation (RAG) using Wikipedia, and prompt-based tool learning to generate Python function calls for the Alpha Vantage API. The findings reveal that general-purpose LLMs exhibit serious hallucination behaviors, with domain-specific fine-tuning (FinMA) sometimes degrading general instruction-following abilities. However, RAG significantly improves factuality in knowledge-intensive tasks, and prompt-based tool learning achieves near-perfect accuracy in stock price queries by delegating data retrieval to external APIs. The study concludes that while LLMs alone are unreliable for precise financial data, integrating external tools and retrieval mechanisms is essential for deploying them safely in finance. Limitations include the narrow scope of tasks and the task-specific nature of the mitigation strategies tested.

Research problem

The paper investigates the prevalence and severity of hallucinations in Large Language Models (LLMs) when performing financial tasks. It specifically addresses the lack of empirical evidence regarding how often and to what extent LLMs generate factually incorrect or outdated information in sensitive financial contexts. The problem is framed around the need to assess the reliability of both general-purpose and domain-specific LLMs for tasks requiring high precision, such as explaining financial concepts and querying historical market data.

Core contributions

• Provides an empirical benchmark for LLM hallucination in finance across three tasks: abbreviation recognition, term explanation, and stock price querying.
• Demonstrates that off-the-shelf LLMs suffer from significant hallucinations, including outdated information and factual inconsistencies.
• Shows that multi-task domain-specific fine-tuning (FinMA) can degrade general instruction-following abilities compared to base models.
• Evaluates and compares four mitigation strategies: few-shot learning, DoLa decoding, RAG, and prompt-based tool learning.
• Establishes that RAG and tool learning are highly effective in reducing hallucinations and improving factual accuracy in financial applications.

Method

• Evaluated LLMs on three financial tasks: acronym recognition, term explanation, and stock price querying.
• Used FactScore metric to quantify factual consistency in long-form financial term explanations.
• Implemented RAG using FAISS vector store to retrieve relevant Wikipedia content for acronym and term tasks.
• Applied DoLa decoding by contrasting outputs from different model layers to enhance factual accuracy.
• Developed prompt-based tool learning to generate Python function calls for the Alpha Vantage API for stock prices.
• Measured accuracy, MAE, and integer part correctness for stock price prediction tasks.
• Compared zero-shot, few-shot, and tool-augmented performance across various model sizes and types.

Results & findings

• GPT-4 achieved 82.5% accuracy on acronym recognition and 90.4% on stock symbol recognition in zero-shot settings.
• FinMA-7B underperformed its base model Llama1-7B, indicating that domain-specific fine-tuning may harm general instruction following.
• Zero-shot Llama2 models had extremely high MAE (~$6300) in stock price queries, showing severe unreliability.
• GPT-3.5 and GPT-4 abstained from answering stock price questions in zero-shot settings, avoiding hallucination but limiting utility.
• RAG improved Llama2-7B-chat's FactScore from 64.68% to 90.48% in term explanation tasks.
• Prompt-based tool learning allowed Llama2-7B-chat to achieve 91.2% accuracy in stock price queries, and GPT-4 achieved 100%.
• DoLa decoding showed limited effectiveness, particularly when models lacked foundational knowledge in their training data.
• Few-shot learning improved format adherence but did not significantly enhance factual precision over instruction-tuned baselines.

Data & experiments

• Models evaluated: Llama2-7B, Llama2-7B-chat, Llama2-13B, Llama2-13B-chat, GPT-3.5-turbo, GPT-4, and FinMA-7B.
• Task 1: Financial abbreviation recognition using 192 acronyms from Wikipedia and 1215 stock symbols from EODData.
• Task 2: Financial term explanation using 160 obscure terms from Wikidata, evaluated with FactScore metric.
• Task 3: Historical stock price querying for 70 tickers on 4 dates, using Alpha Vantage API for ground truth.
• Mitigation methods tested: Few-shot prompting, DoLa decoding, RAG with Wikipedia/FAISS, and tool learning for API calls.
• Metrics used: Accuracy, FactScore, Mean Absolute Error (MAE), and integer part correctness.

Limitations & caveats

• The study focuses on only three financial tasks, which may not represent the full spectrum of financial applications.
• Mitigation strategies are task-specific; their effectiveness in other financial contexts is not guaranteed.
• The stock price query task uses historical data, so it does not test real-time prediction capabilities.
• The evaluation of RAG relies on Wikipedia, which may not be the most authoritative source for all financial data.
• The study does not explore the impact of hallucinations on downstream financial decision-making or portfolio performance.
• Domain-specific fine-tuning results (FinMA) are limited to one model, potentially lacking generalizability.

Curator notes

Discovery status: round2_promoted. Source links: A Comprehensive Review of Generative AI in Finance; Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models; FinBERT: A Pretrained Language Model for Financial Communications; FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models; Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk