Ascending Labs Research
Benchmark checker Law, finance, health, science

Is InsClaimBench still meaningful?

Status:New / unproven Checked against the report, 5 November 2026

Verdict

Promising. It shows that per-rule accuracy hides case-level failures.

Why new / unproven: An academic team released it in October 2026, so outside review has not started.[1][1] InsClaimBench, arXiv 2610.09671 Primary

What “New / unproven” means

Known problems

  • Atomic-rule accuracy reached 95.48%, but exact match across a whole rule vector peaked at 36.90%.[1][1] InsClaimBench, arXiv 2610.09671 Primary
  • It has no claimant dialogue, and it states no jurisdiction.[1][1] InsClaimBench, arXiv 2610.09671 Primary

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 1, Benchmarks

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

  1. [1]
    InsClaimBench, arXiv 2610.09671. It tests insurance claims adjudication at rule and case level. Primary arxiv.org Chapter 1, source 104
  • HealthBench Status:Still useful Yes, as a template. But the dataset has not changed since August 2025.
  • HLAB-Vals 1.1 Status:Still useful Yes. All-pass scoring is strict, so models still spread out.
  • Vals Finance Agent Status:Still useful Yes, with care. The top five sit within 0.7 points, so ranks mean little.
  • Aithos LARA Status:New / unproven Unproven. It is small and has no published lawyer review of its second study.