Benchmark checker Speech and voice

Is Big Bench Audio still meaningful?

Saturated Checked against the report, 5 November 2026

No. The top model scores 99.7%, so it cannot separate frontier speech models.

Why saturated: The chapter rates Big Bench Audio Saturated. StepAudio 3 Realtime scores 99.7% (Nov 2026).^{[1][1] Artificial Analysis, speech-to-speech Primary}

What “Saturated” means

Known problems

It uses synthetic speech in 23 voices, in English only.^{[2][2] Big Bench Audio dataset card Primary}
Single-turn skill tests overstate deployed conduct. On τ-voice, voice agents kept only 30–45% of their text-mode success.^{[3][3] τ-voice, arXiv 2603.13686 Primary}

What to look at instead

Conversational Dynamics, a Full-Duplex-Bench subset, also sits near the ceiling at 98.9% (Nov 2026).^{[1][1] Artificial Analysis, speech-to-speech Primary}^{[4][4] Full-Duplex-Bench, arXiv 2503.04721 Primary}

τ-voice Status:New / unproven Yes. Voice agents keep only 30–45% of text-mode success, so it separates them well.

Where it appears in the report

Chapter 7, Multimodal, voice and embodied

Figure 1. Tests Beyond Text Fill Up in Months, Too
Table 1. Each modality has one number that tells its story
Speech reasoning is solved, but voice agents lose most of their text-mode success
Table 10. Single-turn speech tests have saturated; multi-turn voice agents remain the frontier in “Go deeper: Speech-to-speech and voice-agent benchmarks”

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Artificial Analysis, speech-to-speech. Primary artificialanalysis.ai Chapter 7, source 9
[2]
Big Bench Audio dataset card. Primary huggingface.co Chapter 7, source 57
[3]
τ-voice, arXiv 2603.13686. Primary arxiv.org Chapter 7, source 10
[4]
Full-Duplex-Bench, arXiv 2503.04721. Primary arxiv.org Chapter 7, source 58