Is OSWorld-Verified still meaningful?

Known problems

• About 15% of its tasks need only a terminal, so it overstates screen skills.^{[2][2] What OSWorld tells us Epoch AI Primary}
• OSWorld-Verified fixed community-reported issues in July 2025. The last tagged OSWorld release dates from June 2024.^{[3][3] OSWorld README XLANG Lab Primary}

What to look at instead

• OSWorld 2.0 Status:Still useful Yes. Full-credit scores stay low, though partial credit tells a kinder story.
• Remote Labor Index Status:Still useful Yes. It is one of the few tests where the best agent still fails most tasks.

Where it appears in the report

Chapter 1, Benchmarks

• The new frontier prices tasks in money and hours
• Table 2. First model to cross each threshold in “Go deeper: The full saturation timeline (14 benchmarks)”
• Table 3. How each area replaced its saturated benchmarks, oldest first in “Go deeper: Lineage of replacements, by area”
• Table 7. General and computer-use agent benchmarks, November 2026 in “Go deeper: General and computer-use agent benchmarks (18)”

Chapter 3, Tools and technology

• Eval environments and RL training environments merged into one layer
• Table 4. GitHub API and pypistats, 5 Nov 2026 in “Go deeper: All nine environment projects, plus the benchmark worlds”