Benchmark checker Agents and computer use

Is BrowseComp still meaningful?

Near ceiling Checked against the report, 5 November 2026

Fading. The top score is above 90%, and memory may inflate results.

Why near ceiling: The chapter rates it Near ceiling: GPT-6 Astra scores 91.5% in OpenAI's own table (Sep 2026).^{[1][1] GPT-6 Astra benchmarks explained Vellum Secondary}

What “Near ceiling” means

Known problems

LiveBrowseComp reports search scores 25 to 40 points lower than on BrowseComp (single source).^{[2][2] LiveBrowseComp, arXiv 2605.28721 Primary}
OpenAI stopped updating simple-evals, the BrowseComp reference code, in July 2025.^{[3][3] simple-evals GitHub Primary}^{[4][4] simple-evals README OpenAI Primary}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 1, Benchmarks

Table 5. Reasoning and knowledge benchmarks: what they measure, who maintains them, the top score on 5 November 2026 and the known flaw in “Go deeper: Reasoning and knowledge benchmarks (17)”
Figure 14. OpenAI and Anthropic share only five headline benchmarks

Chapter 3, Tools and technology

Two harnesses carry model benchmarking, and Harbor carries agent tasks
Figure 3. Well-Known Eval Projects Freeze Every Few Months
Table 2. GitHub API and pypistats, 5 Nov 2026 in “Go deeper: All 14 model-benchmark harnesses, with maintainers and status”

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
GPT-6 Astra benchmarks explained, Vellum. It transcribes OpenAI's launch table and discusses its framing. Secondary vellum.ai Chapter 1, source 14
[2]
LiveBrowseComp, arXiv 2605.28721. It argues that memory inflates BrowseComp scores. Primary arxiv.org Chapter 1, source 37
[3]
simple-evals, GitHub. It gives stars and push data. Primary github.com Chapter 3, source 18
[4]
simple-evals README, OpenAI. It announces the end of updates in July 2025. Primary github.com Chapter 3, source 41