DeepFinLLM: an intelligent financial advisor unleashing strategic insights with large language models

In one sentence

Summary

The paper addresses the limitations of traditional rule-based financial tools and static AI models, which often suffer from low accuracy, high latency, and an inability to handle complex, unstructured natural language queries in real-time. DeepFinLLM is positioned as a hybrid system that combines sparse rule-based analysis with dense LLM-driven retrieval to deliver precise, contextually rich financial insights. The system aims to bridge the gap between semantic understanding and dynamic market data, ensuring that users receive timely and relevant advice for investment decisions, risk assessment, and market trend analysis. The methodology employs a multi-layered architecture featuring a presentation layer, state management for conversational continuity, a data processing core, and an API integration layer that fetches live data via the Financial Modeling Prep (FMP) API. The system utilizes the DeepSeek Chat V3 model for semantic processing and intent classification, augmented by NLP techniques like tokenization and named entity recognition. Evaluation was conducted using five benchmark datasets (FinQA, FiQA, NumerSense, FinSim, ConvFinQA) against baselines including FinGPT, BloombergGPT, LLaMA-7B, and RoBERTa-Fin. Metrics included general language model performance (Perplexity, BLEU, ROUGE-L) and finance-specific accuracy, exact match, and inference latency. Results indicate that DeepFinLLM achieves a 94.2% accuracy rate with an inference latency of 0.8 seconds, significantly outperforming baselines in both language generation quality and financial precision. It demonstrated superior performance in factual consistency (96.5%) and ranking quality (nDCG 0.95). The system is particularly useful for wealth advisory and personal investing, offering an intuitive interface for both novices and experts. However, limitations include potential overfitting to benchmark datasets, lack of statistical significance testing against baselines, and unproven generalizability to emerging markets or non-English queries. The authors note that real-world ecological validity and temporal stability require further validation.

Research problem

Traditional financial analysis tools rely on rigid, rule-based algorithms that struggle with complex, multi-dimensional queries and often achieve only 70-75% accuracy. While modern AI and LLMs offer semantic understanding, they frequently lack real-time data integration, leading to outdated insights and high latency. The paper addresses the need for a system that combines the precision of rule-based methods with the adaptability of LLMs and the timeliness of live market data to provide accurate, low-latency financial advice.

Core contributions

• Proposes DeepFinLLM, a hybrid financial advisory system integrating DeepSeek Chat V3 with real-time FMP API data.
• Designs a multi-layered architecture including state management for conversational continuity and asynchronous API integration.
• Develops a comprehensive evaluation framework combining general language metrics (PPL, BLEU) with finance-specific metrics (Accuracy, nDCG).
• Demonstrates superior performance over baselines like FinGPT and BloombergGPT in accuracy (94.2%) and latency (0.8s).
• Provides a detailed analysis of threats to validity, including internal, external, and construct validity concerns.

Method

• Integrates DeepSeek Chat V3 for semantic processing and intent classification.
• Uses Financial Modeling Prep (FMP) API for real-time data retrieval.
• Employs asynchronous API calls with caching to minimize latency.
• Implements a finite state machine for state management and conversational continuity.
• Applies NLP techniques including tokenization, POS tagging, and NER.
• Evaluates performance using standard NLP metrics (PPL, BLEU, ROUGE-L) and financial metrics (Accuracy, nDCG).
• Uses cosine similarity for query expansion and user preference adaptation.

Results & findings

• DeepFinLLM achieves 94.2% accuracy and 0.8-second inference latency.
• Factual consistency score is 96.5%, indicating high reliability.
• Outperforms baselines in Perplexity (3.2 vs 3.4-3.8) and BLEU (0.85 vs 0.82-0.83).
• Achieves an nDCG score of 0.95, demonstrating strong ranking quality.
• Hallucination rate is low at 1.2%.
• User study shows 90% of users find the interface intuitive and valuable.
• Significantly reduces response time compared to traditional minute-scale delays.

Data & experiments

• Utilizes five benchmark datasets: FinQA, FiQA, NumerSense, FinSim, and ConvFinQA.
• Compares DeepFinLLM against four baselines: FinGPT, BloombergGPT, LLaMA-7B, and RoBERTa-Fin.
• Evaluates general language model performance using Perplexity, Fluency, BLEU, ROUGE-L, METEOR, and BERTScore.
• Assesses finance-specific accuracy using Exact Match, F1 Score, nDCG, and Inference Latency.
• Conducts user feedback analysis indicating 90% of users value the fast, reliable insights.

Limitations & caveats

• Potential overfitting to benchmark datasets like FinQA and ConvFinQA.
• Limited generalizability to emerging markets, niche assets, or non-English queries.
• Lack of statistical significance testing in comparisons with baselines.
• Subjective nature of some evaluation metrics (e.g., Explainability Score).
• Controlled evaluation environment may not reflect real-world noisy data conditions.
• Temporal validity is limited to the specific market conditions at the time of evaluation.

Curator notes

Discovery status: institutional_trading_focus_promoted. Source links: large language models robo advisor portfolio advice