Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training

In one sentence

Summary

The paper addresses the challenge of deploying LLMs in finance, where domain-specific terminology, numerical reasoning, and factual accuracy are critical. It argues that in specialized verticals, performance is driven more by the quality and verifiability profile of post-training data than by architectural scaling. The authors propose a data-centric approach, introducing a two-stage hierarchy: using high-quality, verified Chain-of-Thought (CoT) data for Supervised Fine-Tuning (SFT) to establish a robust reasoning foundation, and employing difficulty- and verifiability-aware sampling for Reinforcement Learning (RL) to push generalization on hard tasks without introducing noise from unverified long-form outputs. To implement this, the authors construct ODA-Fin-SFT-318k by aggregating 697k raw samples from 25+ open-source repositories, applying semantic deduplication, synthesizing CoT traces using a large reasoning model, and filtering via length-adaptive verification. For RL, they curate ODA-Fin-RL-12k by selecting samples with high failure rates (>50%) on the SFT model but retaining only those with concise ground truths (<16 tokens) to ensure reliable online verification using a lightweight reward model. Training is conducted on Qwen3-8B using standard SFT and Group Relative Policy Optimization (GRPO) pipelines, with a hybrid reward function combining format adherence and semantic correctness. Evaluated on nine benchmarks spanning general financial understanding, sentiment analysis, and numerical reasoning, ODA-Fin-RL-8B achieves a 74.6% average score, surpassing all open-source financial LLMs of comparable size and matching the larger Qwen3-32B. Ablation studies reveal that training exclusively on distilled CoT data outperforms mixing raw data or adding general math/reasoning samples, which cause negative transfer. The RL stage provides consistent gains, particularly in numerical reasoning (FinQA, TaTQA) and agent-level tasks (Finova), validating that hard-but-verifiable data selection effectively refines instruction-following and complex reasoning capabilities beyond what SFT alone can achieve.

Research problem

The paper tackles the difficulty of adapting LLMs to the financial domain, characterized by dense terminology, strict numerical reasoning requirements, and low tolerance for hallucinations. It questions the prevailing model-centric scaling paradigm, positing that data quality and the difficulty/verifiability profile of post-training data are the primary determinants of performance in specialized verticals. The study aims to disentangle the contributions of SFT data quality versus RL data selection strategies.

Core contributions

• Introduction of a multi-stage distillation and verification protocol to transform raw financial Q&A into high-quality Chain-of-Thought supervision.
• Development of a difficulty- and verifiability-aware sampling strategy for RL, balancing task hardness with reliable online verification.
• Release of ODA-Fin-SFT-318k and ODA-Fin-RL-12k datasets, constructed entirely from open-source resources.
• Empirical demonstration that data-centric post-training can outperform larger and specialized open-source financial LLMs.
• Ablation studies showing that exclusive use of distilled CoT data is superior to mixing raw data or adding general reasoning samples.

Method

• Semantic deduplication using Qwen3-Embedding-8B to ensure data diversity in the raw pool.
• CoT synthesis via Qwen3-235B-A22B-Thinking for samples lacking reasoning traces.
• Length-adaptive verification: using CompassVerifier-7B for short answers and Qwen3-235B-Instruct for long-form logical coherence.
• Token length filtering to exclude samples exceeding 16,384 tokens.
• RL data selection based on empirical failure rate (>50%) on the SFT model, restricted to short ground truths (<16 tokens).
• Hybrid reward function in RL: product of format reward (tag presence) and outcome multiplier (correctness via CompassVerifier-7B).
• Group Relative Policy Optimization (GRPO) with 4 rollouts, KL divergence coefficient of 0.001.
• Ablation studies comparing raw data, partial CoT, full distilled CoT, and augmented CoT (math/table) strategies.

Results & findings

• ODA-Fin-RL-8B achieves a 74.6% average score across nine benchmarks, the highest among open-source 8B-scale models.
• Performance matches Qwen3-32B (74.7%), a model four times larger in parameter scale.
• Outperforms specialized financial LLMs like DianJin-R1-7B (70.3%) and FinMA-7B-full.
• Significant gains in numerical reasoning: 89.3% on TaTQA, surpassing Qwen3-32B by 4.2 points.
• Strong performance on Finova (54.6%), indicating improved agent-level financial reasoning.
• Ablation shows raw data training degrades performance on capable base models (Qwen3-8B), while full distilled CoT is optimal.
• Adding general math or table-calculation data to SFT causes negative transfer and performance drops.
• RL stage yields consistent improvements (+2.5% average) over SFT, particularly in Finova and FinQA.

Data & experiments

• Aggregation of 697k raw samples from 25+ open-source financial repositories, including sentiment, QA, and reasoning datasets.
• Construction of ODA-Fin-SFT-318k via semantic deduplication, CoT synthesis, and length-adaptive verification.
• Curated ODA-Fin-RL-12k dataset focusing on hard-but-verifiable samples with concise ground truths.
• Evaluation on nine benchmarks: FinEval, Finova, FinanceIQ (general); FOMC, FPB, Headlines (sentiment); FinQA, TaTQA, ConvFinQA (numerical).
• Comparison against five open-source financial LLMs (FinMA, Xuanyuan, Plutus, Fin-R1, DianJin-R1) and general models (Qwen3-32B).
• Ablation studies on SFT data composition (raw vs. partial CoT vs. full distilled CoT vs. augmented with math/table data).

Limitations & caveats

• Datasets are constructed entirely from open-source resources, potentially lacking proprietary or real-time market data.
• RL verification is restricted to short ground truths, limiting applicability to complex, open-ended financial analysis.
• The study focuses on 8B-scale models; results may not generalize to larger or smaller architectures.
• Negative transfer observed when adding general math/table data suggests high sensitivity to data distribution shifts.
• Evaluation relies on static benchmarks; real-world deployment risks (e.g., latency, cost) are not addressed.
• The 'hard-but-verifiable' constraint may exclude many valuable but complex financial reasoning tasks.

Curator notes

Discovery status: round1_promoted_seed_for_round2. Source links: A Comparative Analysis of Instruction Fine-Tuning Large Language Models for Financial Text Classification; BloombergGPT: A Large Language Model for Finance; ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering; Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learning; FinBen: A Holistic Financial Benchmark for Large Language Models; FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Mode