DianJin-R1: Evaluating and Enhancing Financial Reasoning in Large Language Models

In one sentence

Summary

The paper addresses the challenge of effective reasoning in large language models for the financial domain, where tasks require domain-specific knowledge, precise numerical calculations, and strict adherence to compliance rules. The authors propose DianJin-R1, a framework that combines reasoning-augmented supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO) reinforcement learning. The core contribution is the construction of DianJin-R1-Data, a high-quality dataset synthesized from CFLUE, FinQA, and a proprietary Chinese Compliance Check (CCC) corpus. The data construction process involves filtering for difficulty and ambiguity, converting multiple-choice questions to open-ended formats, and using multi-agent systems to generate complex reasoning paths for compliance scenarios, verified by GPT-4o. This ensures the training data contains verified, structured reasoning steps alongside final answers. The training methodology consists of two stages. First, models (DianJin-R1-7B and 32B, based on Qwen2.5) undergo SFT to learn generating structured outputs with <think> and <answer> tags. Second, GRPO is applied using hard cases from the dataset, employing dual reward signals: a format reward for strict structural adherence and an accuracy reward for correct final answers. The models are evaluated on five benchmarks: three financial (CFLUE, FinQA, CCC) and two general reasoning (MATH-500, GPQA-Diamond). The experimental design compares DianJin-R1 against non-reasoning baselines (e.g., Qwen2.5, GPT-4o) and other reasoning models (e.g., DeepSeek-R1, QwQ-32B), measuring accuracy and computational efficiency. Results show that DianJin-R1 models consistently outperform their non-reasoning counterparts, particularly on complex financial tasks. DianJin-R1-32B achieved 86.74% on CFLUE, 80.82% on FinQA, and 96.00% on CCC, surpassing even larger general reasoning models like DeepSeek-R1 on financial benchmarks. Notably, on the CCC compliance task, the single-call DianJin-R1 models matched or exceeded the performance of multi-agent systems that required an average of 8.15 API calls per instance, demonstrating superior cost-efficiency. The paper highlights that while financial reasoning training improves general reasoning, it does not fully close the gap with models trained on general reasoning data. Limitations include the exclusion of the proprietary CCC data from public release and the observation that RL improvements were less pronounced on English datasets (FinQA) due to the Chinese-centric RL training data.

Research problem

Large language models struggle with financial reasoning tasks that demand domain-specific knowledge, numerical precision, and regulatory compliance. Existing general reasoning models often underperform on specialized financial benchmarks or incur high computational costs when using multi-agent architectures. The paper aims to develop a scalable, efficient framework to enhance financial reasoning capabilities without relying on expensive multi-step inference.

Core contributions

• Proposes DianJin-R1, a reasoning-enhanced framework combining SFT and GRPO for financial LLMs.
• Constructs DianJin-R1-Data, a high-quality dataset synthesized from CFLUE, FinQA, and proprietary CCC data.
• Introduces a multi-agent workflow for generating complex reasoning paths in compliance scenarios.
• Demonstrates that single-call reasoning models can match multi-agent systems in compliance checking.
• Shows that financial reasoning training improves general reasoning capabilities, though not to the level of general-domain reasoning models.

Method

• Data filtering: Length, difficulty (using small LLMs), and ambiguity (using GPT-4o).
• Reasoning synthesis: DeepSeek-R1 for CoT generation, GPT-4o for verification.
• Multi-agent workflow: Specialized agents for each compliance condition node.
• SFT: Structured output format with <think> and <answer> tags.
• RL: GRPO with format reward (structural adherence) and accuracy reward (answer correctness).
• Evaluation: Rule-based extraction for multiple-choice, GPT-4o for open-ended/numeric answers.

Results & findings

• DianJin-R1-32B achieved 86.74% on CFLUE, surpassing DeepSeek-R1 (86.64%).
• DianJin-R1-32B achieved 96.00% on CCC, matching multi-agent systems with 1 API call vs 8.15.
• SFT alone improved CFLUE accuracy from 69.37% to 75.48% (7B model).
• RL provided additional gains on CFLUE (to 80.32%) and CCC (to 94.50%).
• Training on CFLUE alone yielded significant gains across all test sets.
• General reasoning models like DeepSeek-R1 did not always outperform on financial benchmarks (e.g., dropped on FinQA).

Data & experiments

• Datasets: CFLUE (Chinese financial exams), FinQA (English numerical reasoning), CCC (Chinese compliance dialogues).
• Benchmarks: CFLUE, FinQA, CCC, MATH-500, GPQA-Diamond.
• Baselines: Qwen2.5 variants, GPT-4o, DeepSeek-V3, DeepSeek-R1, QwQ-32B.
• Metrics: Accuracy on test sets, computational cost (API calls).
• Experimental design: Two-stage training (SFT then RL), ablation studies on dataset contributions.

Limitations & caveats

• CCC data is proprietary and not publicly available, limiting reproducibility of that specific task.
• RL improvements were less effective on English datasets (FinQA) due to Chinese-centric RL training data.
• Financial reasoning training does not fully close the gap with models trained on general reasoning data.
• The accuracy reward was only applied to multiple-choice questions in the current version.
• Future work needed for tool-augmented reasoning and fine-grained reward shaping.

Curator notes

Discovery status: round3_promoted. Source links: Benchmarking Large Language Models on CFLUE - A Chinese Financial Language Understanding Evaluation Dataset; BizFinBench: A Business-Driven Real-World Financial Benchmark for Evaluating LLMs; Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning