Is Open ASR Leaderboard still meaningful?

Known problems

• The results file needs a login, so top word error rates rest on secondary sources.^{[4][4] Apidog, open-source speech-to-text models Secondary}
• Public test sets invite "benchmark fitting". A model that scores better on noisy references has learned their spelling quirks.^{[5][5] Open ASR Leaderboard, app.py Primary}
• LibriSpeech no longer counts in the default average (2 Nov 2026).^{[1][1] Open ASR Leaderboard, init.py (version log) Primary}
• Vendor numbers on private sets do not compare. Cekura reports 2.28% for Nemotron 3.5 Transcribe on its own set.^{[6][6] Cekura website Vendor}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 7, Multimodal, voice and embodied

• Figure 1. Tests Beyond Text Fill Up in Months, Too
• Table 1. Each modality has one number that tells its story
• The Open ASR Leaderboard fights benchmark gaming
• Figure 7. In six months the speech leaderboard rebuilt its default score
• Table 9. Top word error rates rest on secondary sources, because the leaderboard's results file needs a login in “Go deeper: Speech recognition and synthesis”
• Figure 11. Robotics, GUI agents and deepfake detection have the weakest independent checks