Benchmark checker Law, finance, health, science

Is Harvey Legal Agent Bench (LAB) still meaningful?

Contested Checked against the report, 5 November 2026

Only with the host named. The same model scores more than twice as high on another board.

Why contested: Scores swing two to three times with the host and harness. Claude Opus 5.5 scores 3.75%, 8.3% or 9.17%, by host.^{[1][1] Harvey LAB v1 Vals AI Primary}^{[2][2] Claude Opus 5.5 system card Anthropic Vendor}^{[3][3] HLAB-Vals-1.1 Vals AI Primary}

What “Contested” means

Known problems

It states no jurisdiction.^{[2][2] Claude Opus 5.5 system card Anthropic Vendor}
Claude Opus 5.5 scores 3.75% on the LAB v1 board, 8.3% on Artificial Analysis's holdout and 9.17% on HLAB-Vals 1.1.^{[1][1] Harvey LAB v1 Vals AI Primary}^{[2][2] Claude Opus 5.5 system card Anthropic Vendor}^{[3][3] HLAB-Vals-1.1 Vals AI Primary}

What to look at instead

HLAB-Vals 1.1 Status:Still useful Yes. All-pass scoring is strict, so models still spread out.

Where it appears in the report

Chapter 1, Benchmarks

The harness and the host can move a score more than the model
Figure 7. The same model gets very different scores in different harnesses
Table 9. Legal skill benchmarks, November 2026 in “Go deeper: Legal skill benchmarks (10)”

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Harvey LAB v1, Vals AI. It shows the older board of October 2026. Primary vals.ai Chapter 1, source 77
[2]
Claude Opus 5.5 system card, Anthropic. It reports capability, cyber, alignment and evaluation-awareness results. Vendor anthropic.com Chapter 1, source 38
[3]
HLAB-Vals-1.1, Vals AI. It ranks models on Harvey's remediated 120-task private holdout. Primary vals.ai Chapter 1, source 74
[4]
Harvey LAB public set, BenchLM. It lists self-reported LAB scores. Secondary benchlm.ai Chapter 1, source 76