Is MMLU-Pro still meaningful?

Known problems

• No 2026 frontier scores come from a primary source.^{[1][1] MMLU-Pro Artificial Analysis Primary}

What to look at instead

GPQA Diamond and then HLE replaced it in the knowledge line.^{[2][2] Humanity's Last Exam site Primary}

• GPQA Diamond Status:Near ceiling Barely. Top models sit within one point of the 97% ceiling, inside the noise.
• Humanity's Last Exam (HLE) Status:Flawed Use with care. Nearly half of a sampled set was defective, so top scores partly reflect broken items.
• HLE-Diamond Status:New / unproven Promising but unproven. It launched in September 2026 and has no independent audit yet.

Where it appears in the report

Chapter 1, Benchmarks

• Table 3. How each area replaced its saturated benchmarks, oldest first in “Go deeper: Lineage of replacements, by area”
• Table 5. Reasoning and knowledge benchmarks: what they measure, who maintains them, the top score on 5 November 2026 and the known flaw in “Go deeper: Reasoning and knowledge benchmarks (17)”