Is Humanity's Last Exam (HLE) still meaningful?

Known problems

• Epoch AI found 12 impossible questions and 10 false negatives in a 48-item sample.^{[1][1] HLE benchmark review Epoch AI Primary}
• HLE-Verified marked 689 of its 2,500 items as uncertain.^{[3][3] HLE-Verified, arXiv 2602.13964 Primary}
• Anthropic reports 67.7% with tools for Claude Opus 5.5 (vendor), against 54.8% for the best no-tools run.^{[2][2] Benchmark data bundle Epoch AI Primary[4][4] Introducing Claude Opus 5.5 Anthropic Vendor}
• Anthropic's 64.4% no-tools figure for Opus 5.5 sits about ten points above Epoch's own runs of Anthropic models.^{[5][5] Claude Opus 5.5 system card Anthropic Vendor}
• FutureHouse estimated in 2025 that 29% of HLE biology and chemistry answers were wrong. Critics disputed the method.^{[6][6] HLE answer errors The Decoder Secondary}

What to look at instead

The maintainers launched HLE-Diamond in September 2026, because frontier models neared the noise ceiling.^{[7][7] Humanity's Last Exam site Primary}

• HLE-Diamond Status:New / unproven Promising but unproven. It launched in September 2026 and has no independent audit yet.

Where it appears in the report

Chapter 1, Benchmarks

• Figure 1. Each New Benchmark Fills Up Faster Than the Last
• Figure 4. Best score on benchmarks that still resist, November 2026
• Figure 5. Thirteen Flawed verdicts in ten weeks
• Figure 6. In several famous benchmarks, a third to a half of audited tasks are broken
• Table 4. Epoch AI Benchmark Review verdicts and the main defects found, August to October 2026 in “Go deeper: Epoch's verdicts, benchmark by benchmark”
• Table 5. Reasoning and knowledge benchmarks: what they measure, who maintains them, the top score on 5 November 2026 and the known flaw in “Go deeper: Reasoning and knowledge benchmarks (17)”
• Figure 14. OpenAI and Anthropic share only five headline benchmarks

Chapter 3, Tools and technology

• Two harnesses carry model benchmarking, and Harbor carries agent tasks

Chapter 6, China and Asia

• Chinese model cards report agentic scores and no safety evaluations
• Figure 3. Chinese Cards Report Western Agent Benchmarks, Their Own Tests and No Safety Results