Benchmark checker Agents and computer use

Is OSWorld 2.0 still meaningful?

Still useful Checked against the report, 5 November 2026

Yes. Full-credit scores stay low, though partial credit tells a kinder story.

Why still useful: Claude Opus 5 completes 31.4% of tasks in full, so frontier models still have room to climb.^{[1][1] Benchmark data bundle Epoch AI Primary}

What “Still useful” means

Known problems

OpenAI reports 72.6% for GPT-6 Astra on an offline set (vendor).^{[3][3] GPT-6 Astra benchmarks explained Vellum Secondary}
Qwen3.8-Flash-Next reports its best of two harnesses.^{[4][4] Qwen3.8-Flash-Next-FP8 model card Primary} MiniMax-M3.1-Flash re-runs rivals in-house and reports its own numbers.^{[5][5] MiniMax-M3.1-Flash model card Primary}

What to look at instead

Version 2.1 followed, but Epoch AI rates it Flawed.^{[6][6] OSWorld 2.1 review Epoch AI Primary}

OSWorld 2.1 Status:Flawed No. Grading errors could affect half of the score.

Where it appears in the report

Chapter 1, Benchmarks

Figure 1. Each New Benchmark Fills Up Faster Than the Last
Figure 4. Best score on benchmarks that still resist, November 2026
The new frontier prices tasks in money and hours
Table 7. General and computer-use agent benchmarks, November 2026 in “Go deeper: General and computer-use agent benchmarks (18)”
Figure 14. OpenAI and Anthropic share only five headline benchmarks

Chapter 6, China and Asia

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Benchmark data bundle, Epoch AI. Dated scores (5 Nov 2026) and ceilings for most benchmarks here. Primary epoch.ai Chapter 1, source 1
[2]
OSWorld 2.0. It describes the long-horizon successor to OSWorld. Primary osworld-v2.xlang.ai Chapter 1, source 54
[3]
GPT-6 Astra benchmarks explained, Vellum. It transcribes OpenAI's launch table and discusses its framing. Secondary vellum.ai Chapter 1, source 14
[4]
Qwen3.8-Flash-Next-FP8 model card. It reports Alibaba's benchmarks and its best-of-harness rule. Primary huggingface.co Chapter 6, source 34
[5]
MiniMax-M3.1-Flash model card. It reports MiniMax's benchmarks, including an in-house reproduction. Primary huggingface.co Chapter 6, source 38
[6]
OSWorld 2.1 review, Epoch AI. Impossible tasks, false negatives and false positives. Primary epoch.ai Chapter 1, source 8