Benchmark checker Vision and generation

Is HallusionBench still meaningful?

Still useful Checked against the report, 5 November 2026

Yes. Frontier models still score only 60–67, so it separates them on hallucination.

Why still useful: Frontier models scored only 60.4–66.7 in late 2025, while document tests passed 95.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary}^{[2][2] HallusionBench, arXiv 2310.14566 Primary} The chapter calls hallucination the clearest unsolved vision test.

What “Still useful” means

Known problems

The chapter found no 2026 primary score, so the latest figures date from November 2025.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary}
Most late-2025 vision scores come from one primary table, the Qwen3-VL report.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 7, Multimodal, voice and embodied

Figure 1. Tests Beyond Text Fill Up in Months, Too
Table 1. Each modality has one number that tells its story
Vision tests filled up, and the vision arena can no longer rank the frontier
Table 2. Image and document benchmarks with their best scores in “Go deeper: Image and document benchmarks, full table”
Figure 11. Robotics, GUI agents and deepfake detection have the weakest independent checks

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Qwen3-VL technical report, arXiv 2511.21631. Lab report; Table 2 compares four model families. Primary arxiv.org Chapter 7, source 1
[2]
HallusionBench, arXiv 2310.14566. Primary arxiv.org Chapter 7, source 26