Benchmark checker Law, finance, health, science

Is Vals Finance Agent still meaningful?

Still useful Checked against the report, 5 November 2026

Yes, with care. The top five sit within 0.7 points, so ranks mean little.

Why still useful: The chapter lists it as active, and its 337-question test set stays private forever.^{[1][1] Finance Agent benchmark Vals AI Primary}

What “Still useful” means

Known problems

The top five sit within 0.7 points; v2.1 changed the judge jury and grading prompt.^{[2][2] Finance Agent v2.1 Vals AI Primary}
It tests analyst skill, not conduct rules.^{[1][1] Finance Agent benchmark Vals AI Primary}
Vals AI uses a subject-pays model. Its CEO compares it to "a student paying to take the SAT".^{[4][4] Vals AI profile Carly Secondary}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 1, Benchmarks

Domain benchmarks test expert skill, not lawful conduct
Table 11. Finance benchmarks, November 2026 in “Go deeper: Finance benchmarks (7)”

Chapter 4, Market and companies

Leaderboards became businesses, and every business model carries a conflict of interest
Figure 12. Only the Buyer-Paid Benchmark Avoids Both Conflicts
Table 16. Business models and their conflicts in full in “Go deeper: Leaderboard companies, conflict table and more detail”

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Finance Agent benchmark, Vals AI. It describes the private test set, the judge and v1.1 scores. Primary vals.ai Chapter 1, source 98
[2]
Finance Agent v2.1, Vals AI. It gives v2.1 scores and describes the new judge jury. Primary vals.ai Chapter 1, source 100
[3]
Finance Agent Benchmark, arXiv 2508.00828. It describes the expert questions and tools. Primary arxiv.org Chapter 1, source 99
[4]
Vals AI profile, Carly. It quotes the Vals CEO on who pays for tests. Secondary usecarly.com Chapter 4, source 120