Benchmark checker Law, finance, health, science

Is EU-Agent-Bench still meaningful?

New / unproven Checked against the report, 5 November 2026

Thin. It is a useful single-turn probe, but nobody maintains it.

Why new / unproven: It is small and single-turn: 600 samples, with only the first tool call checked. Its repository saw its last push in August 2025.^{[1][1] EU-Agent-Bench, arXiv 2510.21524 Primary}^{[2][2] EU-Agent-Bench repository GitHub Primary}

What “New / unproven” means

Known problems

It is single-turn; the repository saw its last push in August 2025.^{[2][2] EU-Agent-Bench repository GitHub Primary}
Reviewers note missing tax, labour and financial rules.^{[3][3] Paper notes on EU-Agent-Bench Secondary}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 1, Benchmarks

Domain benchmarks test expert skill, not lawful conduct
Figure 11. Few benchmarks test conduct under a named law
Table 10. EU law, regulation and privacy benchmarks by tier: mapping, assessment or conduct, November 2026 in “Go deeper: EU law, regulation and privacy benchmarks (8)”
Several risks still have no good public benchmark

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
EU-Agent-Bench, arXiv 2510.21524. It measures illegal agent tool calls under EU law. Primary arxiv.org Chapter 1, source 86
[2]
EU-Agent-Bench repository, GitHub. It shows the date of the last push. Primary github.com Chapter 1, source 87
[3]
Paper notes on EU-Agent-Bench. It summarises the sample design and reviewer comments. Secondary en.papernotes.org Chapter 1, source 18