Ascending Labs Research
← Home
Benchmark checker

47 of 82 benchmarks in this report are saturated, near their ceiling or flawed

Look up any benchmark to see whether it still means something in November 2026. Each check gives a verdict, the best published score, the known problems and what to use instead.

Snapshot 5 November 2026 · Every fact comes from a chapter, with its source · How we decide the status

Status
Domain

Saturated tests sit near 100%; tests that still work leave room at the top

Each dot is one benchmark in this report, in a lane for its status. Labels mark the best-known tests. Matches for your search and filters stay dark.

Hover or tap a dot. Arrow keys move between dots.

Still useful22 benchmarksNew / unproven10 benchmarksContested3 benchmarksNear ceiling14 benchmarksSaturated17 benchmarksFlawed16 benchmarks0255075100%Not in report
Data table: all 82 benchmarks
Every benchmark with its status, best published score and release date
BenchmarkStatusBest scoreReleased
AgentDojoStill usefulASR 0.33—
AgentHarmStill useful0.30 violation rate—
Agents' Last ExamFlawedAbout 26%—
Aithos LARANew / unproven59.2%—
ARC-AGI-1Saturated98.5%2019
ARC-AGI-2Near ceiling95.0%Mar 2025
ARC-AGI-3Contested62.7%Mar 2026
BEHAVIOR-1KStill useful12.4%2024
BFCL v4Flawed——
Big Bench AudioSaturated99.7%—
BrowseCompNear ceiling91.5%—
C-EvalSaturated92.1–93.1May 2023
Chatbot Arena (LMArena)Contested——
CMMLUSaturated90.0Jun 2023
CMT-BenchmarkFlawed——
CritPtFlawed——
CursorBenchStill useful57.8%—
CybenchSaturated93%Aug 2024
Deepfake-Eval-2024Still useful—Mar 2025
DeepSWEFlawed74.1%—
DocVQASaturated96.5–97.1—
EU-Agent-BenchNew / unprovenAbout 9 of 20—
EuroEvalStill usefulgpt-6-astra first—
ExploitBenchNear ceiling100%—
ForesightSafety BenchNew / unproven—Feb 2026
FrontierCodeNew / unproven54.6%—
FrontierMath ErdősStill useful4.4%Sep 2026
FrontierMath Tier 4Saturated100%Jun 2026
FrontierMath Tiers 1–3Near ceiling93.7%—
GDPval-AAStill useful1846 Elo—
GenEvalSaturated96.7%—
GenEval 2New / unproven35.8%Dec 2025
GPQA DiamondNear ceiling96.4%Nov 2023
GSONear ceiling88.2%—
HallusionBenchStill useful60.4–66.72023
HarmBenchSaturatedNear 0–2%—
Harvey Legal Agent Bench (LAB)Contested23.7%—
HealthBenchStill useful60%May 2025
HealthBench ProfessionalFlawed73.4%—
HLAB-Vals 1.1Still useful27.08%—
HLE-DiamondNew / unproven59.9%Sep 2026
Humanity's Last Exam (HLE)Flawed54.8%Jan 2025
IMO 2026Saturated42/42—
IndicSafeNew / unproven—Mar 2026
InsClaimBenchNew / unproven36.90%Oct 2026
Lech Mazur WritingFlawed——
LegalBenchSaturated88.56%—
LIBEROFlawedAbove 90%2023
MASKNear ceiling96.28—
MATH Level 5Saturated98.1%2021
MathArenaStill useful84.0%—
MedQASaturated——
METR time horizonSaturatedAbout 17 hours—
MMLUSaturated88.1%2020
MMLU-ProSaturated89.8%—
MMLU-ProXStill useful——
MMMUFlawed80.6–84.2—
MMMU-ProNear ceiling85.5%Sep 2024
NAVSIM v2Still usefulabout 87.5 EPDMSJun 2025
nuScenes open-loop planningFlawed——
Open ASR LeaderboardStill useful——
OSWorld 2.0Still useful31.4%—
OSWorld 2.1Flawed81.8%—
OSWorld-VerifiedNear ceiling72.1%Jul 2025
OTIS Mock AIME 2024-25Saturated100%Dec 2024
Remote Labor IndexStill useful21.25%Oct 2025
ScreenSpot-ProNear ceiling92.7%Apr 2025
SimpleQA VerifiedStill useful76.0%—
SuperCLUEStill useful72.622023
SWE-bench ProFlawed61.5—
SWE-bench VerifiedFlawed83.5%Aug 2024
Terminal-Bench 4.0Flawed67.2%—
TextQuestsFlawed——
Vals Finance AgentStill useful67.85%—
Vending-Bench 2Still useful$16,484—
Video-MMENear ceiling85.1—
Virology Capabilities Test (VCT)Still useful43.8%—
WMDPNear ceiling0.91 (Bio)—
XSTest and OR-BenchNear ceiling——
τ-voiceNew / unproven79.9%2026
τ²-benchNear ceiling87.9%—
τ³-BankingNew / unproven55.2%Mar 2026
Status and scores as the chapters give them on 5 November 2026. Scores that are not percentages, and benchmarks without a release date in the report, sit in the right-hand column. Source: the eight chapters of State of AI Evals 2026 and their sources.

82 benchmarks, A to Z

  1. AgentDojo Yes, run it yourself. The public results page stopped in February 2025. Status:Still useful Safety and security ASR 0.33Grok 4.20
  2. AgentHarm Yes, as part of the standard stack. It uses simulated tools only. Status:Still useful Safety and security 0.30 violation rateGrok 4.20
  3. Agents' Last Exam Use with care. Shortcuts exist in 22 of 50 audited tasks, and lab scores disagree. Status:Flawed Agents and computer use About 26%
  4. Aithos LARA Unproven. It is small and has no published lawyer review of its second study. Status:New / unproven Law, finance, health, science 59.2%Claude Opus 4.7
  5. ARC-AGI-1 No. Many models now match the human score, so it cannot rank them. Status:Saturated Knowledge and reasoning 98.5%many models
  6. ARC-AGI-2 Barely. It went from 54% to 95% in nine months, and little headroom is left. Status:Near ceiling Knowledge and reasoning 95.0%GPT-6 Astra (max)
  7. ARC-AGI-3 Yes, but only with the harness named. One adapter moves the top score by 37 points. Status:Contested Knowledge and reasoning 62.7%GPT-6 Astra
  8. BEHAVIOR-1K Yes. The best challenge entry succeeds on only 12.4% of tasks, which leaves much room. Status:Still useful Robotics and driving 12.4%NeurIPS 2025 challenge winner
  9. BFCL v4 No. Epoch AI rates it Flawed, and one task rewards declining the call. Status:Flawed Agents and computer use No score in the report
  10. Big Bench Audio No. The top model scores 99.7%, so it cannot separate frontier speech models. Status:Saturated Speech and voice 99.7%StepAudio 3 Realtime
  11. BrowseComp Fading. The top score is above 90%, and memory may inflate results. Status:Near ceiling Agents and computer use 91.5%GPT-6 Astra
  12. C-Eval No. A 2026 base model scores above 92, and labs now cite it only in base-model tables. Status:Saturated Chinese and Asian 92.1–93.1DeepSeek-V4.1-Flash (base model)
  13. Chatbot Arena (LMArena) Partly. It still shows crowd preference, but private testing and unequal data access distort the ranking. Status:Contested Knowledge and reasoning No score in the report
  14. CMMLU No. A 2026 base model scores 90.0, so CMMLU no longer separates frontier models. Status:Saturated Chinese and Asian 90.0ERNIE-5.5-Flash-Base
  15. CMT-Benchmark No. Epoch AI found flaws in 30 of 50 sampled items. Status:Flawed Knowledge and reasoning No score in the report
  16. CritPt No. Epoch AI found flaws in at least 21 of its 71 challenges. Status:Flawed Knowledge and reasoning No score in the report
  17. CursorBench Yes, as one signal. Remember that a product company owns it. Status:Still useful Coding 57.8%Claude Opus 5.5 (max)
  18. Cybench No. It went from 50% to 90% in five months, and Anthropic now calls it saturated. Status:Saturated Safety and security 93%Claude Opus 4.6
  19. Deepfake-Eval-2024 Yes, as a reality check. Open-source detectors lose about half their skill on its real fakes. Status:Still useful Vision and generation No score in the report
  20. DeepSWE Partly. Its fresh tasks resist contamination, but Epoch AI found issues in at least 23 of 113. Status:Flawed Coding 74.1%GPT-6 Astra (xhigh)
  21. DocVQA Only as a regression check. Top models score about 97%, so it no longer separates them. Status:Saturated Vision and generation 96.5–97.1Qwen3-VL-235B
  22. EU-Agent-Bench Thin. It is a useful single-turn probe, but nobody maintains it. Status:New / unproven Law, finance, health, science About 9 of 20Gemini-2.5-Flash
  23. EuroEval Yes, for European language skill. It does not test legal conduct. Status:Still useful Multilingual gpt-6-astra firstGPT-6 Astra
  24. ExploitBench Fading fast. A vendor reports a perfect score, though Epoch AI rates the test Verified. Status:Near ceiling Safety and security 100%GPT-6 Astra
  25. ForesightSafety Bench Too early to say, because it is new and the report found no published model results. Status:New / unproven Chinese and Asian No score in the report
  26. FrontierCode Unproven. Labs headline it, but nobody has confirmed who runs it or how. Status:New / unproven Coding 54.6%Claude Opus 5.5
  27. FrontierMath Erdős Yes. It still leaves almost all of its room at the top, though the set is small. Status:Still useful Maths 4.4%GPT-6.1 Sol
  28. FrontierMath Tier 4 No. A model solved every problem three months after version 2 launched. Status:Saturated Maths 100%GPT-6.1 Sol
  29. FrontierMath Tiers 1–3 Fading. The top score is 93.7%, and the funder can see part of the set. Status:Near ceiling Maths 93.7%GPT-6.1 Sol
  30. GDPval-AA Yes, with care. Aggregators print it on different scales. Status:Still useful Agents and computer use 1846 EloClaude Opus 5.5
  31. GenEval No. Models score near 97%, and its automatic judge has drifted away from human judgment. Status:Saturated Vision and generation 96.7%Gemini 2.5 Flash Image
  32. GenEval 2 Yes, tentatively. The best model reaches only 35.8%, but outside use is still thin. Status:New / unproven Vision and generation 35.8%
  33. GPQA Diamond Barely. Top models sit within one point of the 97% ceiling, inside the noise. Status:Near ceiling Knowledge and reasoning 96.4%Gemini 4 Argon
  34. GSO Fading. The top score is 88.2%, and the scaffold drives results. Status:Near ceiling Coding 88.2%Claude Fable 5.1
  35. HallusionBench Yes. Frontier models still score only 60–67, so it separates them on hallucination. Status:Still useful Vision and generation 60.4–66.7
  36. HarmBench No longer for the frontier. Refusal violations sit near zero, and nobody maintains it. Status:Saturated Safety and security Near 0–2%
  37. Harvey Legal Agent Bench (LAB) Only with the host named. The same model scores more than twice as high on another board. Status:Contested Law, finance, health, science 23.7%Kimi K3.1 Pro
  38. HealthBench Yes, as a template. But the dataset has not changed since August 2025. Status:Still useful Law, finance, health, science 60%o3
  39. HealthBench Professional No. Half of the sampled tasks had clear errors, and boards disagree on scoring. Status:Flawed Law, finance, health, science 73.4%Claude Opus 5
  40. HLAB-Vals 1.1 Yes. All-pass scoring is strict, so models still spread out. Status:Still useful Law, finance, health, science 27.08%Gemini 4 Argon
  41. HLE-Diamond Promising but unproven. It launched in September 2026 and has no independent audit yet. Status:New / unproven Knowledge and reasoning 59.9%GPT-6 Astra
  42. Humanity's Last Exam (HLE) Use with care. Nearly half of a sampled set was defective, so top scores partly reflect broken items. Status:Flawed Knowledge and reasoning 54.8%GPT-6 Astra
  43. IMO 2026 No. Systems score full marks, and most lab claims are self-graded. Status:Saturated Maths 42/42Huawei Celia and dots-note-3.0
  44. IndicSafe Promising but early: one 2026 study shows safety verdicts rarely carry across Indic languages. Status:New / unproven Chinese and Asian No score in the report
  45. InsClaimBench Promising. It shows that per-rule accuracy hides case-level failures. Status:New / unproven Law, finance, health, science 36.90%
  46. Lech Mazur Writing No. Epoch AI rates it Flawed. Status:Flawed Knowledge and reasoning No score in the report
  47. LegalBench No. The top ten models sit within 1.6 points, and it covers US law only. Status:Saturated Law, finance, health, science 88.56%Claude Fable 5
  48. LIBERO No. High scores reflect memorised action sequences and collapse when the scene shifts slightly. Status:Flawed Robotics and driving Above 90%several VLAs
  49. MASK Fading. Top scores sit near 96, partly because evasive answers count as honest. Status:Near ceiling Safety and security 96.28claude-opus-4-6 (non-thinking)
  50. MATH Level 5 No. Top models score 98.1%, so it cannot separate them. Status:Saturated Maths 98.1%GPT-5
  51. MathArena Yes. Fresh contests limit contamination, but small contests give wide error bars. Status:Still useful Maths 84.0%GPT-6.1 Sol
  52. MedQA No. It is saturated, and exam questions match few real clinical queries. Status:Saturated Law, finance, health, science No score in the report
  53. METR time horizon No longer. The best agents outran the task suite, and updates stopped in September 2026. Status:Saturated Agents and computer use About 17 hoursClaude Mythos Preview
  54. MMLU No. MMLU stopped separating frontier models years ago, and labs no longer report it. Status:Saturated Knowledge and reasoning 88.1%GPT-4o
  55. MMLU-Pro No. Top scores sit near 90%, and no 2026 frontier results come from a primary source. Status:Saturated Knowledge and reasoning 89.8%Gemini 3 Pro Preview
  56. MMLU-ProX Yes, for language gaps. It does not rank the frontier. Status:Still useful Multilingual No score in the report
  57. MMMU No. Its test answers went public in February 2026, so new scores may reflect contamination. Status:Flawed Vision and generation 80.6–84.2
  58. MMMU-Pro Partly. Scores sit near 85% and differ by source, so compare models within one source only. Status:Near ceiling Vision and generation 85.5%GPT-6.1 Terra
  59. NAVSIM v2 Yes. It is the main public driving benchmark, and its hard split stays at 34–37. Status:Still useful Robotics and driving about 87.5 EPDMS
  60. nuScenes open-loop planning No. Baselines without any perception compete, so its scores say little about driving. Status:Flawed Robotics and driving No score in the report
  61. Open ASR Leaderboard Yes, its new default resists gaming with private data. LibriSpeech alone no longer counts there. Status:Still useful Speech and voice No score in the report
  62. OSWorld 2.0 Yes. Full-credit scores stay low, though partial credit tells a kinder story. Status:Still useful Agents and computer use 31.4%Claude Opus 5
  63. OSWorld 2.1 No. Grading errors could affect half of the score. Status:Flawed Agents and computer use 81.8%Claude Opus 5.5
  64. OSWorld-Verified No longer for the frontier. Agents have reached the human baseline. Status:Near ceiling Agents and computer use 72.1%Claude Sonnet 4.6
  65. OTIS Mock AIME 2024-25 No. Several models score 100%, and 30 items are too few to separate them. Status:Saturated Maths 100%several models, incl. GPT-6.1 Terra
  66. Remote Labor Index Yes. It is one of the few tests where the best agent still fails most tasks. Status:Still useful Agents and computer use 21.25%Claude Opus 5.5
  67. ScreenSpot-Pro Barely. Top models score 92.7%, and the test covers static screens only. Status:Near ceiling Vision and generation 92.7%GPT-6 Astra
  68. SimpleQA Verified Yes. It passed Epoch AI's audit, though it rewards guessing over saying "I don't know". Status:Still useful Knowledge and reasoning 76.0%Gemini 4 Argon
  69. SuperCLUE Yes for Chinese-market ranking, since private questions resist contamination, but scores from different rounds do not compare. Status:Still useful Chinese and Asian 72.62Qwen3.8-Max-0902
  70. SWE-bench Pro No. About a third of its tasks are broken, and agents have cheated on it. Status:Flawed Coding 61.5Muse Spark 1.1
  71. SWE-bench Verified No. Its tests reject correct fixes, models have seen the answers, and labs dropped it. Status:Flawed Coding 83.5%Claude Opus 4.7
  72. Terminal-Bench 4.0 Use with care. It is a launch-table favourite, but it is rated Flawed and lab numbers disagree. Status:Flawed Coding 67.2%Kimi K3.1 Pro
  73. TextQuests No. Epoch AI rates it Flawed. Status:Flawed Agents and computer use No score in the report
  74. Vals Finance Agent Yes, with care. The top five sit within 0.7 points, so ranks mean little. Status:Still useful Law, finance, health, science 67.85%Claude Opus 5.5
  75. Vending-Bench 2 Yes, as a long-horizon signal. One run per model gives wide error bars. Status:Still useful Agents and computer use $16,484Gemini 4 Carbon
  76. Video-MME Barely. Top scores reached 85 in 2025, and no dated 2026 leaderboard exists. Status:Near ceiling Vision and generation 85.1Gemini 2.5 Pro
  77. Virology Capabilities Test (VCT) Yes, as a warning signal. A high paper score may not mean real-world uplift. Status:Still useful Safety and security 43.8%o3
  78. WMDP Fading. Models now score well above the human baseline on its biology items. Status:Near ceiling Safety and security 0.91 (Bio)Grok 4.20
  79. XSTest and OR-Bench Only as a hygiene check. Static prompts are easy to tune for. Status:Near ceiling Safety and security No score in the report
  80. τ-voice Yes. Voice agents keep only 30–45% of text-mode success, so it separates them well. Status:New / unproven Speech and voice 79.9%Phonic 1 (High)
  81. τ²-bench Fading. Top scores are high, and the newer τ³-Banking spreads models more. Status:Near ceiling Agents and computer use 87.9%Qwen3.5-397B-A17B
  82. τ³-Banking Promising. The best agent scores 55.2%, but outside review is thin. Status:New / unproven Agents and computer use 55.2%Qwen 3.8 Max

How we decide the status

Each benchmark gets one of six statuses. The words follow the status table in chapter 1, Benchmarks. If several apply, the first wins, in this order: Flawed, Saturated, Near ceiling, Contested, New / unproven, Still useful. Each check gives its reason with a source.

Status:Still useful 22
Frontier models still spread out, and the maintainer still publishes results.
Status:New / unproven 10
Released in 2026, or small and thinly tested. It has no audit, little outside review or no public leaderboard yet.
Status:Contested 3
Published scores disagree by more than the gap between models, because the harness, host, scoring or source differs.[1][1] GPT-6 Astra on ARC-AGI-3 ARC Prize Primary[2][2] HLAB-Vals-1.1 Vals AI Primary
Status:Near ceiling 14
Top scores sit within a few points of the ceiling or of human level.
Status:Saturated 17
Top scores sit at or near the ceiling, so the benchmark no longer ranks frontier models. In 2026 this often took less than a year.[3][3] Benchmark data bundle Epoch AI Primary
Status:Flawed 16
Epoch AI's Benchmark Review found defects that could affect at least 20% of the score.[4][4] Benchmark Review registry Epoch AI Primary[5][5] HLE benchmark review Epoch AI Primary We also use it when another published audit found broken, leaked or gameable items on a similar scale.

Sources for this page

The definitions above cite these sources. Each benchmark page lists its own.

  1. [1]
    GPT-6 Astra on ARC-AGI-3, ARC Prize. The provider adapter and the 99.9% result. Primary arcprize.org Chapter 1, source 9
  2. [2]
    HLAB-Vals-1.1, Vals AI. It ranks models on Harvey's remediated 120-task private holdout. Primary vals.ai Chapter 1, source 74
  3. [3]
    Benchmark data bundle, Epoch AI. Dated scores (5 Nov 2026) and ceilings for most benchmarks here. Primary epoch.ai Chapter 1, source 1
  4. [4]
    Benchmark Review registry, Epoch AI. Lists 13 Flawed, 5 Verified and 8 unrated benchmarks in November 2026. Primary epoch.ai Chapter 1, source 3
  5. [5]
    HLE benchmark review, Epoch AI. The Flawed verdict, the sample defects and the review rubric. Primary epoch.ai Chapter 1, source 4