Outside text, tests fill up fast and almost nobody checks the numbers independently

Every modality follows the same arc, from saturated tests to thin independent checks

AI now sees, draws, speaks, listens, drives and moves robots. Each modality has its own tests, arenas and failure modes, but the story repeats.

First-generation tests saturate within one or two model generations. Vision "Pro" tests, GUI grounding and speech reasoning all neared their ceilings within 12–24 months.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary[9][9] Artificial Analysis, speech-to-speech Primary[17][17] BenchLM, ScreenSpot-Pro Secondary} Human-preference arenas then take over. Soon they also stop separating the frontier, because the top models sit inside the noise.^{[4][4] Arena, vision leaderboard Primary}

The hard part is the same in every modality. Real conditions, real tasks and an independent evaluator are scarce. Deepfake detectors lose about half their skill on real fakes.^{[8][8] Deepfake-Eval-2024, arXiv 2503.02857 Primary} Robot policies that ace a simulator fail when the camera moves.^{[13][13] LIBERO-Plus, arXiv 2510.13626 Primary} Voice agents lose most of their text-mode success once callers speak with noise and accents.^{[10][10] τ-voice, arXiv 2603.13686 Primary}

Vision tests filled up, and the vision arena can no longer rank the frontier

Vision-language models answer questions about images, documents, charts and video. Their evaluation has split into saturated regression checks, harder successors and preference arenas.

Saturated perception tests stay in technical reports only as regression checks. Most late-2025 scores come from one primary table, the Qwen3-VL report. It compares Qwen3-VL with Gemini 2.5 Pro, GPT-5 and Claude Opus 4.1.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary} DocVQA reached 96.5–97.1, ChartQA 90.3 and MathVista-mini 85.8. Even the authors of ChartQA's successor call ChartQA saturated.^{[20][20] ChartQAPro, arXiv 2504.05506 Primary}

Harder successors replaced them, and most now near their own ceiling. MMMU-Pro removed questions that text-only models could answer. It raised the options to ten and embedded some questions in the image. At launch, models scored 16.8–26.9 points lower than on MMMU.^{[23][23] MMMU-Pro, arXiv 2409.02813 Primary} Within two years, aggregator scores passed 85%.^{[2][2] BenchLM, MMMU-Pro Secondary} Models still stay "close to random guesses" on rare image types, such as music sheets and chemical structures.^{[22][22] MMMU benchmark site Primary}

Hallucination remains the clearest unsolved test. Frontier models scored only 60.4–66.7 on HallusionBench in late 2025, while document tests passed 95.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary[26][26] HallusionBench, arXiv 2310.14566 Primary} Video understanding also lacks a current, dated leaderboard. Video-MME tops out at 85.1 and VideoMMMU at 84.6, both from November 2025 tables.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary[28][28] Video-MME, arXiv 2405.21075 Primary}

Preference arenas rank models by votes, but the frontier sits inside the noise. Arena, formerly LMArena, runs the largest vision arena. On 19 October 2026 its board held 1,493,948 votes across 166 models.^{[4][4] Arena, vision leaderboard Primary} The whole top ten fits inside 16 points, with overlapping intervals. A gap that small does not support any ranking claim.

Other arenas have gone quiet. WildVision-Arena collected real-user preferences in 2024, and its GPT-4 judge reached a Spearman correlation of 0.94 with arena Elo.^{[34][34] WildVision, arXiv 2406.11069 Primary} Its Space has not changed since 22 December 2024.^{[35][35] Hugging Face API, WildVision Space Primary} The OpenCompass Open VLM Leaderboard last changed on 11 March 2026.^{[36][36] Hugging Face API, Open VLM Leaderboard Primary}

GUI grounding has reached its ceiling

Perception no longer limits GUI agents. ScreenSpot-Pro tests grounding on 1,581 instructions across 23 professional applications and three operating systems.^{[16][16] ScreenSpot-Pro, arXiv 2504.07981 Primary} The best model scored 18.9% at launch in April 2025. By November 2026, BenchLM listed GPT-6 Astra at 92.7%, Claude Opus 4.8 at 87.9% and GPT-5.4 at 85.4%.^{[17][17] BenchLM, ScreenSpot-Pro Secondary} The test covers static screens only, with no state tracking or recovery. Reliability, harness effects and broken task checkers now limit GUI agents instead.

Blind human votes now rank generation, because automatic judges drift

Image, video and music generation have no single right answer. Evaluators therefore rely on human preference, and automatic metrics serve as cheap proxies.

A frozen judge falls behind the models it grades. GenEval checks objects, counts, colours and positions in text-to-image output. Its Meta FAIR authors show that GenEval "has drifted far from human judgment over time".^{[5][5] GenEval 2, arXiv 2512.16853 Primary} A large human study confirmed the saturation. The same drift affects LLM judges in text. Any judge-model approach needs regular re-calibration against fresh human labels.

A harder successor resets the scale. GenEval 2 adds more primitives and deeper composition, scored per primitive with Soft-TIFA.^{[5][5] GenEval 2, arXiv 2512.16853 Primary} The best model reached only 35.8% at prompt level in December 2025 (single source).^{[37][37] alphaXiv overview of GenEval 2 Single source} Learned scorers such as HPSv2 and HPSv3 also serve as proxies. HPSv2 fine-tunes CLIP on 798,090 human choices, and HPSv3 uses a VLM and 1.17 million comparisons.^{[39][39] HPSv2, arXiv 2306.09341 Primary[40][40] HPSv3, arXiv 2508.03789 Primary} Any fixed scorer falls behind newer models.

Two arenas dominate, and they disagree on video. Arena collects votes from an open crowd. Artificial Analysis (AA) pays vetted private evaluators. Both fit votes with a Bradley-Terry model on an Elo scale.^{[48][48] Artificial Analysis, image methodology Primary} On images they agree: GPT-Image-2.5 Sunburst leads both boards.^{[45][45] Arena, text-to-image leaderboard Primary[46][46] Artificial Analysis, text-to-image leaderboard Primary} On video they split. Gemini Omni 1.1 Flash ranks first on Arena but tenth on AA.^{[6][6] Arena, text-to-video leaderboard Primary[7][7] Artificial Analysis, text-to-video leaderboard Primary} Neither source explains the gap. Likely causes include the voter pool, the prompt set and the inclusion of audio.

Audio metrics lack validation against human perception. Fréchet Audio Distance (FAD) remains the default metric for music quality. Few studies test its link to perceived quality. Its critics name three biases: sample size, embedding choice and reference set.^{[44][44] Adapting FAD for music, arXiv 2311.01616 Primary} Music Arena, live since July 2025, runs pairwise votes for text-to-music systems.^{[47][47] Music Arena, arXiv 2507.20900 Primary}

Deepfake detectors lose about half their skill on real fakes

Detection benchmarks built in the lab overstate how well detectors work on fakes found online.

Deepfake-Eval-2024 gives the clearest picture of real-world detection. Its authors collected in-the-wild deepfakes from 2024. The set holds 45 hours of video, 56.5 hours of audio and 1,975 images. The items come from 88 websites in 52 languages.^{[8][8] Deepfake-Eval-2024, arXiv 2503.02857 Primary} AUC, the area under the curve, measures how well a detector separates real from fake. Open-source state-of-the-art detectors lost 45–50% of their AUC against academic benchmarks.

Commercial and fine-tuned detectors did better, but not well enough. They "do not yet reach the accuracy of deepfake forensic analysts".^{[8][8] Deepfake-Eval-2024, arXiv 2503.02857 Primary} Transparency rules, such as Article 50 of the EU AI Act, will need in-the-wild test sets. We found no standing, independent leaderboard for deepfake detection.

Speech reasoning is solved, but voice agents lose most of their text-mode success

Speech recognition, speech synthesis and voice agents each have their own boards. The gap between skill on a test and conduct on a call is widest for agents.

Single-turn speech tests have saturated. StepAudio 3 Realtime scores 99.7% on Big Bench Audio, a set of 1,000 spoken reasoning questions.^{[9][9] Artificial Analysis, speech-to-speech Primary[57][57] Big Bench Audio dataset card Primary} It also scores 98.9% on turn-taking dynamics such as pauses, backchannels and interruptions.^{[58][58] Full-Duplex-Bench, arXiv 2503.04721 Primary} Big Bench Audio uses synthetic speech in 23 voices, in English only. AA's Speech-to-Speech Index puts Phonic 1 (High) at 87.1, just ahead of Grok Voice 3.0 at 86.9.^{[9][9] Artificial Analysis, speech-to-speech Primary}

The τ-voice benchmark shows the gap between skill and conduct most clearly. Its authors extend τ²-bench to full-duplex voice calls in retail, airline and telecom.^{[10][10] τ-voice, arXiv 2603.13686 Primary[60][60] ICML 2026 poster for τ-voice Primary} Each of 278 tasks checks the final database state. In text mode, GPT-5 with reasoning reaches 85% pass@1. Voice agents reach 31–51% with clean audio and 26–38% with realistic noise and accents. The agent's own behaviour causes 79–90% of the failures.

Voice-testing vendors sell quality assurance to builders, not ratings to buyers. Coval, Hamming, Cekura, Bluejay and Cyara combine simulated callers, LLM judges, latency metrics and production monitoring.^{[63][63] Coval website Vendor[65][65] Hamming website Vendor[54][54] Cekura website Vendor[66][66] Bluejay website Vendor[67][67] Cyara website Vendor} Hamming claims 95%+ agreement with human evaluators. Nobody has audited such claims. Cyara's "EU AI Act conformity check" comes closest to a compliance product.

The Open ASR Leaderboard fights benchmark gaming

The Open ASR Leaderboard is the reference board for speech recognition. Its paper (v4, 30 March 2026) benchmarks 86 open and closed systems on 12 datasets.^{[49][49] Open ASR Leaderboard paper, arXiv 2510.06961 Primary} It reports word error rate (WER) for accuracy and inverse real-time factor for speed. During 2026, its maintainers rebuilt the default average around cleaned and private data.^{[11][11] Open ASR Leaderboard, init.py (version log) Primary}

Private data and a fitting check now resist gaming. Appen, DataoceanAI and Voice Arena supply private sets, aggregated so that no provider or split leaks.^{[50][50] Open ASR Leaderboard, constants.py Primary} The private tab exists to "address risks of overfitting to public datasets". A "benchmark fitting" tab compares the noisy VoxPopuli split with a cleaned version.^{[51][51] Open ASR Leaderboard, app.py Primary} A model that scores better on the noisy references has learned their spelling quirks. The maintainers call this the "benchmaxing phenomenon".

Top WERs rest on secondary sources, because the results file needs a login. NVIDIA Nemotron 3.5 Transcribe averages 5.33% and ranks ninth, behind eight closed APIs.^{[52][52] Apidog, open-source speech-to-text models Secondary} Qwen3-ASR-1.7B averages 5.40%.^{[52][52] Apidog, open-source speech-to-text models Secondary} Text-to-speech is ranked by preference: AA's Speech Arena names ElevenLabs Eleven v4 top at Elo 1334.^{[55][55] Artificial Analysis, text-to-speech Primary} We found no 2026 standard for mean opinion score (MOS) tests.

Robot scores collapse under small changes, and labs mostly grade themselves

Robotics has the least standardised evaluation of any modality. Simulation is cheap but fragile. Real-robot tests are costly, so labs run them in-house.

LIBERO shows how fragile simulation scores can be. LIBERO-PRO perturbed objects, initial states, instructions and environments.^{[12][12] LIBERO-PRO, arXiv 2510.03827 Primary} Models that scored above 90% collapsed to 0%. They kept grasping when the target changed and ignored corrupted instructions. The authors call this "rote memorization of action sequences". LIBERO-Plus found drops from 95% to below 30% under modest camera and start-state changes.^{[13][13] LIBERO-Plus, arXiv 2510.13626 Primary}

Most robot papers run too few rollouts to support their claims. NVIDIA names four common flaws.^{[76][76] The Robot Report on NVIDIA robot evaluation Secondary} Train and test scenes overlap, and fixed task sets saturate. Binary success hides causes, and rollouts are too few. Seventy rollouts at 90% success leave a confidence band about 15 points wide. A band of ±2 points needs about 1,030 rollouts, using Clopper-Pearson intervals.

Each lab tests its own way, on its own task suite. Google DeepMind runs Gemini Robotics 1.5 comparisons as interleaved A/B/n tests on one real work cell.^{[77][77] Gemini Robotics 1.5, arXiv 2510.03342 Primary} It uses a 230-task benchmark and a continuous "progress score". Physical Intelligence tests π0.5 in "entirely new homes" and reports deployment throughput for π*0.6.^{[79][79] π0.5, arXiv 2504.16054 Primary[80][80] π*0.6, arXiv 2511.14759 Primary} World models now help rank checkpoints. Google DeepMind checked a Veo-based simulator against more than 1,600 real runs.^{[83][83] Gemini Robotics policies in a Veo world simulator, arXiv 2512.10675 Primary} The field accepts such results for ranking, not for absolute success claims.

Independent real-world evaluation started only in 2025 and 2026. RoboArena ran 612 double-blind comparisons across seven universities and ranked policies more accurately than central evaluation.^{[86][86] RoboArena, arXiv 2506.18123 Primary} PhAIL scores real bin picking by units per hour.^{[90][90] Robotics & Automation News on PhAIL Secondary} Robocurve, a three-person YC start-up, says plainly: "Labs evaluate in-house, and no independent group has stepped in".^{[91][91] Y Combinator, Robocurve profile Vendor} The situation resembles text LLM evaluation around 2022.

Driving moved to pseudo-simulation

Open-loop metrics fell first, then the CARLA Leaderboard closed. On nuScenes, simple baselines that use only the car's own state compete without any perception.^{[95][95] Ego status in open-loop driving, arXiv 2312.03031 Primary} The CARLA Leaderboard then closed for good, and version 2.1 had already made scores non-comparable with 2.0.^{[14][14] CARLA Autonomous Driving Leaderboard Primary} NAVSIM v2 now fills the gap with pseudo-simulation on real data. It correlates with closed-loop simulation at R² = 0.8, against 0.7 for the best open-loop method.^{[15][15] NAVSIM v2 pseudo-simulation, arXiv 2506.04218 Primary} Top navtest scores reach about 87.5, against a human level of 90.3 (snippets).^{[98][98] IDOL, arXiv 2605.31476 Primary[99][99] RAP, arXiv 2510.04333 Primary}

Classic ML has a mature, regulator-written method, and it stops at generative AI

Credit scorecards and fraud detectors have been validated the same way since 2011. The method rests on lifecycle review, independence and risk tiers, not on benchmark scores.

SR 26-2 replaced SR 11-7 in April 2026, and it leaves out generative AI. The Federal Reserve issued SR 26-2 on 17 April 2026.^{[101][101] Federal Reserve, SR 26-2 Primary} It supersedes SR 11-7 (2011) and SR 21-8 on anti-money-laundering models. It is "expected to be most relevant" to banks with over $30 billion in assets. Footnote 3 states that generative and agentic AI models "are not within the scope of this guidance".^{[18][18] Federal Reserve, SR 26-2 attachment Primary} The OCC and FDIC issued matching texts (single secondary source).^{[102][102] CRA Insights on SR 26-2 Single source}

The EU AI Act draws its own line between credit and fraud. Annex III lists creditworthiness checks and credit scoring of natural persons as high-risk. It excludes AI that detects financial fraud. It adds risk assessment and pricing in life and health insurance.^{[19][19] EU AI Act, Annex III Primary} A bank's credit model therefore falls under the high-risk rules, but its fraud model does not.

Classic monitoring vendors are moving to agents, and open drift tools are stalling. Arthur's homepage now leads with agent governance and security.^{[106][106] Arthur website Vendor} CrowdStrike agreed to buy Fiddler on 3 November 2026 for agent observability.^{[108][108] CrowdStrike on Fiddler (press release) Primary} NannyML last received a code push in July 2025 and whylogs in January 2025.^{[111][111] GitHub, NannyML Primary[112][112] GitHub, whylogs Primary} Tabular fairness and drift tools now look like a commodity layer. Model-risk specialists such as ValidMind carry classic validation forward.^{[103][103] ValidMind, SR 26-2 use case Vendor}

Outside text and generation, almost nobody checks the numbers independently

Nine lessons hold across all modalities. The last one matters most: independent evaluators are scarce wherever AI leaves the screen.

Every ladder saturates, and the replacement starts the clock again. Vision "Pro" tests, GUI grounding and speech reasoning all neared their ceilings within 12–24 months.^{[1][1] Qwen3-VL technical report, arXiv 2511.21631 Primary[9][9] Artificial Analysis, speech-to-speech Primary[17][17] BenchLM, ScreenSpot-Pro Secondary} Arenas then rank by votes, but tight ranks carry no meaning. The vision top ten spans 16 points, and the video arenas disagree on first place.^{[4][4] Arena, vision leaderboard Primary[6][6] Arena, text-to-video leaderboard Primary[7][7] Artificial Analysis, text-to-video leaderboard Primary}

Real-world tests lag academic ones, and statistics stay thin. Deepfake detectors lose about half their AUC in the wild.^{[8][8] Deepfake-Eval-2024, arXiv 2503.02857 Primary} LIBERO scores collapse under small perturbations.^{[12][12] LIBERO-PRO, arXiv 2510.03827 Primary} Robot papers run too few rollouts, and Artificial Analysis admits its intervals may run narrow.^{[76][76] The Robot Report on NVIDIA robot evaluation Secondary[48][48] Artificial Analysis, image methodology Primary} Lab and aggregator numbers also disagree. MMMU-Pro differs by 3.4 points between Google's page and BenchLM for one model.^{[24][24] Google DeepMind, Gemini Pro page Vendor[2][2] BenchLM, MMMU-Pro Secondary}

Private data and fitting checks are the best defence against gaming. The Open ASR Leaderboard adds private held-out data, cleans references, detects benchmark fitting and verifies contributor claims.^{[11][11] Open ASR Leaderboard, init.py (version log) Primary[51][51] Open ASR Leaderboard, app.py Primary} Frozen judges need the same discipline: GenEval's judge drifted by up to 17.7% from human judgment.^{[5][5] GenEval 2, arXiv 2512.16853 Primary} Every judge model needs scheduled re-calibration on fresh human labels.

Sources

Every number keeps its date and source. Badges mark source quality: primary, secondary, vendor (a claim about the publisher's own product), single source, or estimate.

1. [1]
   Qwen3-VL technical report, arXiv 2511.21631. Lab report; Table 2 compares four model families. Primary arxiv.org
2. [2]
   BenchLM, MMMU-Pro. Aggregator of self-reported scores, 53 models. Secondary benchlm.ai
3. [3]
   BenchLM, CharXiv. Aggregator. Secondary benchlm.ai
4. [4]
   Arena, vision leaderboard. Primary arena.ai
5. [5]
   GenEval 2, arXiv 2512.16853. Primary arxiv.org
6. [6]
   Arena, text-to-video leaderboard. Primary arena.ai
7. [7]
   Artificial Analysis, text-to-video leaderboard. Primary artificialanalysis.ai
8. [8]
   Deepfake-Eval-2024, arXiv 2503.02857. Primary arxiv.org
9. [9]
   Artificial Analysis, speech-to-speech. Primary artificialanalysis.ai
10. [10]
    τ-voice, arXiv 2603.13686. Primary arxiv.org
11. [11]
    Open ASR Leaderboard, init.py (version log). Primary huggingface.co
12. [12]
    LIBERO-PRO, arXiv 2510.03827. Primary arxiv.org
13. [13]
    LIBERO-Plus, arXiv 2510.13626. Primary arxiv.org
14. [14]
    CARLA Autonomous Driving Leaderboard. Primary leaderboard.carla.org
15. [15]
    NAVSIM v2 pseudo-simulation, arXiv 2506.04218. Primary arxiv.org
16. [16]
    ScreenSpot-Pro, arXiv 2504.07981. Primary arxiv.org
17. [17]
    BenchLM, ScreenSpot-Pro. Aggregator, 19 models. Secondary benchlm.ai
18. [18]
    Federal Reserve, SR 26-2 attachment. Primary federalreserve.gov
19. [19]
    EU AI Act, Annex III. Primary artificialintelligenceact.eu
20. [20]
    ChartQAPro, arXiv 2504.05506. Primary arxiv.org
21. [21]
    OCRBench v2, arXiv 2501.00321. Primary arxiv.org
22. [22]
    MMMU benchmark site. Primary mmmu-benchmark.github.io
23. [23]
    MMMU-Pro, arXiv 2409.02813. Primary arxiv.org
24. [24]
    Google DeepMind, Gemini Pro page. Vendor deepmind.google
25. [25]
    Another Wrapper, MMMU-Pro with tools. Aggregator. Secondary anotherwrapper.com
26. [26]
    HallusionBench, arXiv 2310.14566. Primary arxiv.org
27. [27]
    POPE, arXiv 2305.10355. Primary arxiv.org
28. [28]
    Video-MME, arXiv 2405.21075. Primary arxiv.org
29. [29]
    MVBench, arXiv 2311.17005. Primary arxiv.org
30. [30]
    LongVideoBench, arXiv 2407.15754. Primary arxiv.org
31. [31]
    EgoSchema, arXiv 2308.09126. Primary arxiv.org
32. [32]
    The Decoder on Qwen3-VL. Secondary the-decoder.com
33. [33]
    af.net on GPT-5.4 Pro. Unverified claim. Single source af.net
34. [34]
    WildVision, arXiv 2406.11069. Primary arxiv.org
35. [35]
    Hugging Face API, WildVision Space. Primary huggingface.co
36. [36]
    Hugging Face API, Open VLM Leaderboard. Primary huggingface.co
37. [37]
    alphaXiv overview of GenEval 2. Single source alphaxiv.org
38. [38]
    T2I-CompBench++, arXiv 2307.06350. Primary arxiv.org
39. [39]
    HPSv2, arXiv 2306.09341. Primary arxiv.org
40. [40]
    HPSv3, arXiv 2508.03789. Primary arxiv.org
41. [41]
    VBench-2.0, arXiv 2503.21755. Primary arxiv.org
42. [42]
    CreativePro on ToMoviee 2.0 (press release). Vendor creativepro.com
43. [43]
    Hugging Face API, VBench Leaderboard Space. Primary huggingface.co
44. [44]
    Adapting FAD for music, arXiv 2311.01616. Primary arxiv.org
45. [45]
    Arena, text-to-image leaderboard. Primary arena.ai
46. [46]
    Artificial Analysis, text-to-image leaderboard. Primary artificialanalysis.ai
47. [47]
    Music Arena, arXiv 2507.20900. Primary arxiv.org
48. [48]
    Artificial Analysis, image methodology. Primary artificialanalysis.ai
49. [49]
    Open ASR Leaderboard paper, arXiv 2510.06961. Primary arxiv.org
50. [50]
    Open ASR Leaderboard, constants.py. Primary huggingface.co
51. [51]
    Open ASR Leaderboard, app.py. Primary huggingface.co
52. [52]
    Apidog, open-source speech-to-text models. Secondary apidog.com
53. [53]
    NYU Shanghai RITS on Cohere Transcribe. Secondary rits.shanghai.nyu.edu
54. [54]
    Cekura website. Vendor cekura.ai
55. [55]
    Artificial Analysis, text-to-speech. Primary artificialanalysis.ai
56. [56]
    Hugging Face API, TTS Arena V2. Primary huggingface.co
57. [57]
    Big Bench Audio dataset card. Primary huggingface.co
58. [58]
    Full-Duplex-Bench, arXiv 2503.04721. Primary arxiv.org
59. [59]
    Full-Duplex-Bench-v2, arXiv 2510.07838. Primary arxiv.org
60. [60]
    ICML 2026 poster for τ-voice. Primary icml.cc
61. [61]
    VoiceBench, arXiv 2410.17196. Primary arxiv.org
62. [62]
    VoiceBench on GitHub. Primary github.com
63. [63]
    Coval website. Vendor coval.ai
64. [64]
    The Next Web on Coval's Series A. Secondary thenextweb.com
65. [65]
    Hamming website. Vendor hamming.ai
66. [66]
    Bluejay website. Vendor getbluejay.ai
67. [67]
    Cyara website. Vendor cyara.com
68. [68]
    LIBERO, arXiv 2306.03310. Primary arxiv.org
69. [69]
    CALVIN, arXiv 2112.03227. Primary arxiv.org
70. [70]
    RoboCasa, arXiv 2406.02523. Primary arxiv.org
71. [71]
    ManiSkill3, arXiv 2410.00425. Primary arxiv.org
72. [72]
    Habitat 3.0, arXiv 2310.13724. Primary arxiv.org
73. [73]
    BEHAVIOR-1K, arXiv 2403.09227. Primary arxiv.org
74. [74]
    Robot Learning Collective on the BEHAVIOR Challenge. Snippet only. Secondary robot-learning-collective.github.io
75. [75]
    BEHAVIOR Challenge 2025 leaderboard. Snippet only, provisional. Primary behavior.stanford.edu
76. [76]
    The Robot Report on NVIDIA robot evaluation. Reports a vendor benchmark. Secondary therobotreport.com
77. [77]
    Gemini Robotics 1.5, arXiv 2510.03342. Primary arxiv.org
78. [78]
    Gemini Robotics, arXiv 2503.20020. Primary arxiv.org
79. [79]
    π0.5, arXiv 2504.16054. Primary arxiv.org
80. [80]
    π*0.6, arXiv 2511.14759. Primary arxiv.org
81. [81]
    GR00T N1, arXiv 2503.14734. Primary arxiv.org
82. [82]
    SIMPLER, arXiv 2405.05941. Primary arxiv.org
83. [83]
    Gemini Robotics policies in a Veo world simulator, arXiv 2512.10675. Primary arxiv.org
84. [84]
    Moonlight review of the Veo evaluator. Single source themoonlight.io
85. [85]
    GigaWorld-1 and WMBench, arXiv 2607.02642. Primary arxiv.org
86. [86]
    RoboArena, arXiv 2506.18123. Primary arxiv.org
87. [87]
    AutoEval, arXiv 2503.24278. Primary arxiv.org
88. [88]
    RobotArena ∞, arXiv 2510.23571. Primary arxiv.org
89. [89]
    Open X-Embodiment, arXiv 2310.08864. Primary arxiv.org
90. [90]
    Robotics & Automation News on PhAIL. Secondary roboticsandautomationnews.com
91. [91]
    Y Combinator, Robocurve profile. Company self-description. Vendor ycombinator.com
92. [92]
    startuphub.ai on Robocurve. Single source startuphub.ai
93. [93]
    GitHub, LeRobot. Primary github.com
94. [94]
    GitHub, LIBERO. Primary github.com
95. [95]
    Ego status in open-loop driving, arXiv 2312.03031. Primary arxiv.org
96. [96]
    Bench2Drive, arXiv 2406.03877. Primary arxiv.org
97. [97]
    NAVSIM, arXiv 2406.15349. Primary arxiv.org
98. [98]
    IDOL, arXiv 2605.31476. Snippet only. Primary arxiv.org
99. [99]
    RAP, arXiv 2510.04333. Snippet only. Primary arxiv.org
100. [100]
     BenchLM, Claude Fable 5 versus Opus 4.5. Aggregator. Secondary benchlm.ai
101. [101]
     Federal Reserve, SR 26-2. Primary federalreserve.gov
102. [102]
     CRA Insights on SR 26-2. Single source crai.com
103. [103]
     ValidMind, SR 26-2 use case. Vendor docs.validmind.ai
104. [104]
     Evidently AI website. Vendor evidentlyai.com
105. [105]
     GitHub API, Evidently. Primary api.github.com
106. [106]
     Arthur website. Vendor arthur.ai
107. [107]
     GitHub, arthur-engine. Primary github.com
108. [108]
     CrowdStrike on Fiddler (press release). Primary crowdstrike.com
109. [109]
     GitHub, Fairlearn. Primary github.com
110. [110]
     GitHub API, AIF360. Primary api.github.com
111. [111]
     GitHub, NannyML. Primary github.com
112. [112]
     GitHub, whylogs. Primary github.com
113. [113]
     GitHub, alibi-detect. Primary github.com