The List / Categories
Financial question answering combines evidence retrieval, table interpretation and arithmetic. A correct final answer alone may not establish that the model retrieved the right period or used the right units.
Use the RAG guide for retrieval design and the reports guide for document-specific constraints.
The paper introduces FinQA, a large-scale dataset of 8,281 expert-annotated financial question-answer pairs with executable reasoning programs over S&P 500 earnings reports, demonstrating that current LLMs significantly lag behind human experts in complex multi-step numerical reasoning.
The paper introduces TAT-QA, a benchmark for financial question answering over hybrid tabular and textual data, and proposes the TAGOP model which uses sequence tagging and symbolic aggregation operators to achieve 58.0% F1, significantly outperforming baselines but lagging behind human experts.
The WWW'18 Open Challenge established benchmarks for aspect-based financial sentiment analysis and opinion-based question answering, revealing that while sentiment prediction is feasible, aspect classification and opinion QA remain difficult tasks with limited performance.
ConvFinQA introduces a large-scale dataset for conversational financial question answering that requires complex, multi-turn numerical reasoning chains, revealing that both neural symbolic and prompting-based LLMs significantly lag behind human experts in handling long-range dependencies and domain-specific calculations.
FinanceBench is a benchmark of 10,231 financial QA questions from public filings that reveals state-of-the-art LLMs like GPT-4-Turbo and Claude-2 still fail or hallucinate in over 20% of cases even with retrieval or long-context augmentation.
The paper introduces MultiHiertt, a benchmark for numerical reasoning over multi-hierarchical tabular and textual financial data, and proposes MT2Net, a retrieval-reasoning model that significantly outperforms baselines but still lags far behind human experts.
DocFinQA extends the FinQA dataset by grounding 7,437 financial numerical reasoning questions in full-length SEC filings (averaging 123k words), revealing that even state-of-the-art LLMs and retrieval systems struggle significantly with long-context financial document analysis.
BizBench introduces a comprehensive benchmark for evaluating LLMs' quantitative reasoning in finance through program synthesis, quantity extraction, and domain knowledge tasks, revealing that current models are bottlenecked by limited financial background knowledge rather than just code generation capabilities.
FAMMA is a multilingual, multimodal benchmark for advanced financial reasoning that reveals significant gaps in LLMs' domain knowledge and demonstrates that fine-tuning on distilled reasoning trajectories improves performance on unseen financial questions.
DianJin-R1 enhances financial reasoning in LLMs through a structured SFT and GRPO reinforcement learning pipeline on a curated dataset, achieving state-of-the-art accuracy on Chinese financial benchmarks while significantly reducing inference costs compared to multi-agent systems.
FinChain introduces a symbolic benchmark for verifiable multi-step financial reasoning, revealing that even frontier LLMs struggle with complex chain-of-thought verification despite strong final-answer accuracy.
The paper introduces a cognitive fine-tuning framework using a curated MCQ dataset, structured chain-of-thought reasoning, and robustness augmentation to train open-source LLMs for stable financial decision-making, demonstrating that smaller models can approach frontier performance in bullish and mixed market regimes when evaluated via a two-stage static and sequential trading simulation.