Is CritPt still meaningful?
Status:Flawed Checked against the report, 5 November 2026
Verdict
No. Epoch AI found flaws in at least 21 of its 71 challenges.
Why flawed: Epoch AI rated CritPt Flawed on 13 October 2026: at least 21 of 71 challenges are flawed.[1][1] Benchmark Review registry Epoch AI Primary
What “Flawed” meansKnown problems
- At least 21 of 71 challenges are flawed.[1][1] Benchmark Review registry Epoch AI Primary
What to look at instead
The report names no direct successor.
Where it appears in the report
Chapter 1, Benchmarks
- Figure 5. Thirteen Flawed verdicts in ten weeks
- Figure 6. In several famous benchmarks, a third to a half of audited tasks are broken
- Table 4. Epoch AI Benchmark Review verdicts and the main defects found, August to October 2026 in “Go deeper: Epoch's verdicts, benchmark by benchmark”
Sources
Copied from the chapter source lists. Each number links back to the source in its chapter.
- [1] Benchmark Review registry, Epoch AI. Lists 13 Flawed, 5 Verified and 8 unrated benchmarks in November 2026. Primary epoch.ai Chapter 1, source 3