FinVault: Benchmarking Financial Agent Safety in Execution-Grounded Environments

In one sentence

Summary

The paper addresses the critical gap in evaluating the security of LLM-powered financial agents, which currently rely on abstract or content-level safety checks that fail to capture execution-grounded risks in real-world operational workflows. The authors argue that existing benchmarks overlook systemic risks arising from multi-step decision-making, tool invocation, and state-changing actions in highly regulated financial environments. To bridge this gap, they propose FinVault, a benchmark designed to evaluate agent robustness under adversarial conditions that approximate real-world execution, featuring isolated sandboxes with state-writable databases and explicit compliance constraints. FinVault comprises 31 regulatory case-driven scenarios across credit, insurance, securities, and payments, containing 107 predefined vulnerabilities and 963 test cases. The dataset includes 856 attack samples covering prompt injection, jailbreaking, and financially adapted attacks, alongside 107 benign cases for false-positive evaluation. The experimental setup evaluates ten mainstream LLMs and three defense models using metrics like Attack Success Rate (ASR) and False Positive Rate (FPR). The methodology involves a three-stage attack generation pipeline of expert design, model augmentation, and human verification, ensuring real-world plausibility and regulatory relevance. Results show that existing defenses are largely ineffective, with ASRs reaching 50.0% for vulnerable models and 6.7% for the most robust (Claude-Haiku-4.5). Semantic attacks like role-playing and instruction overriding are significantly more effective than technical attacks. The study highlights a trade-off in defense mechanisms, where higher detection rates come with unacceptable false positive rates that disrupt business continuity. Limitations include the exclusion of emerging attack strategies and the simplification of sandbox environments compared to production systems. The paper concludes that general-purpose safety alignment does not transfer well to financial contexts, necessitating financial-specific defenses.

Research problem

The paper addresses the lack of security evaluation benchmarks for LLM-powered financial agents that account for execution-grounded risks in real-world operational workflows. Existing evaluations focus on content compliance or abstract agent settings, failing to capture systemic risks from multi-step decision-making, tool invocation, and state-changing actions in regulated financial environments. The specific problem is the inability to verify whether attacks result in non-compliant business actions or risk consequences due to the absence of state-writable environments and explicit compliance constraints in current benchmarks.

Core contributions

• Proposes FinVault, the first systematic security benchmark for financial agents featuring 31 regulatory case-driven scenarios in isolated sandboxes with state-writable databases.
• Constructs a vulnerability-driven threat model with 107 high-risk vulnerabilities and 963 test cases covering prompt injection, jailbreaking, and financially adapted attacks.
• Demonstrates that existing defense mechanisms are ineffective in realistic financial settings, with high attack success rates even for state-of-the-art models.
• Reveals that semantic attacks (e.g., role-playing, instruction overriding) are significantly more effective than technical attacks, exposing structural weaknesses in current safety alignment.

Method

• Construction of 31 regulatory case-driven scenarios with state-writable business databases and explicit compliance constraints.
• Three-stage attack generation pipeline: expert design, model-based augmentation, and human verification.
• Categorization of 107 vulnerabilities into five types: privilege bypass, compliance violation, information leakage, fraudulent approval, and audit evasion.
• Evaluation of 10 LLMs and 3 defense models using Attack Success Rate (ASR) and False Positive Rate (FPR).
• Analysis of attack effectiveness across eight techniques: direct JSON injection, instruction overriding, role playing, progressive prompting, encoding obfuscation, hypothetical scenarios, authority impersonation, and emotional manipulation.
• Assessment of defense models (LLaMA Guard 3/4, GPT-OSS-Safeguard) for detection performance and operational overhead.
• Case study analysis of representative attack-defense interactions in credit, AML, insurance, and securities scenarios.

Results & findings

• Qwen3-Max exhibits the highest average ASR at 50.0%, while Claude-Haiku-4.5 is the most robust with 6.7% ASR.
• Vulnerability compromise rates reach up to 85.98% for the weakest models, indicating unacceptable security levels for financial applications.
• Role-playing and instruction overriding attacks achieve ASRs up to 64.5% on Qwen3-Max, significantly outperforming technical attacks.
• LLaMA Guard 4 achieves the highest True Positive Rate (61.10%) but suffers from a high False Positive Rate (29.91%), disrupting workflows.
• Insurance service scenarios are the most susceptible to attacks (up to 65.2% ASR), while credit scenarios are less vulnerable due to rule-based processes.
• Claude models show stricter instruction boundary enforcement and conservative tool invocation behavior, contributing to their robustness.
• Semantic attacks exploit the lack of deep financial compliance understanding in general-purpose models.

Data & experiments

• 31 regulatory case-driven financial scenarios across credit, insurance, securities, payments, compliance, and risk management.
• 107 predefined vulnerabilities categorized into privilege bypass, compliance violation, information leakage, fraudulent approval, and audit evasion.
• 963 test cases: 856 attack samples (8 techniques) and 107 benign business samples.
• Evaluation of 10 mainstream LLMs (e.g., Qwen3-Max, Claude-Sonnet-4.5, GPT-4o) and 3 defense models (LLaMA Guard 3/4, GPT-OSS-Safeguard).
• Metrics include Attack Success Rate (ASR), False Positive Rate (FPR), token consumption, and inference time.
• Experimental design includes multi-turn interactions and state-writable databases to verify observable business state changes.

Limitations & caveats

• The benchmark does not cover emerging attack strategies beyond the eight included techniques.
• Sandboxed environments inevitably differ from production financial systems in scale, integration complexity, and operational constraints.
• The ASR metric treats all successful attacks uniformly, ignoring differences in attack severity or downstream impact.
• The study focuses on a limited set of models and may not represent the entire landscape of financial agents.
• The evaluation of defense models is limited to three specific systems, potentially missing other effective approaches.
• The paper does not address the long-term cumulative effects of multi-turn attacks on agent behavior.

Curator notes

Discovery status: regtech_compliance_focus_promoted. Source links: Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk