Benchmark checker Robotics and driving

Is NAVSIM v2 still meaningful?

Still useful Checked against the report, 5 November 2026

Yes. It is the main public driving benchmark, and its hard split stays at 34–37.

Why still useful: NAVSIM v2 is now the main public driving benchmark. It correlates with closed-loop simulation at R² = 0.8.^{[1][1] NAVSIM v2 pseudo-simulation, arXiv 2506.04218 Primary} Its navhard split stays at 34–37 EPDMS.^{[2][2] IDOL, arXiv 2605.31476 Primary}^{[3][3] RAP, arXiv 2510.04333 Primary}

What “Still useful” means

Known problems

The chapter read the top scores only in paper snippets.^{[2][2] IDOL, arXiv 2605.31476 Primary}^{[3][3] RAP, arXiv 2510.04333 Primary}
On navtest, top scores of about 87.5 already sit close to the human level of 90.3.^{[2][2] IDOL, arXiv 2605.31476 Primary}^{[3][3] RAP, arXiv 2510.04333 Primary}

What to look at instead

The CARLA Leaderboard closed for good, and its version 2.1 had already made scores non-comparable with 2.0.^{[4][4] CARLA Autonomous Driving Leaderboard Primary}

Where it appears in the report

Chapter 7, Multimodal, voice and embodied

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
NAVSIM v2 pseudo-simulation, arXiv 2506.04218. Primary arxiv.org Chapter 7, source 15
[2]
IDOL, arXiv 2605.31476. Snippet only. Primary arxiv.org Chapter 7, source 98
[3]
RAP, arXiv 2510.04333. Snippet only. Primary arxiv.org Chapter 7, source 99
[4]
CARLA Autonomous Driving Leaderboard. Primary leaderboard.carla.org Chapter 7, source 14