Is nuScenes open-loop planning still meaningful?

Known problems

• Ego-status baselines without perception compete with full systems.^{[1][1] Ego status in open-loop driving, arXiv 2312.03031 Primary}
• The best open-loop method correlates with closed-loop simulation at R² = 0.7, against 0.8 for NAVSIM v2.^{[2][2] NAVSIM v2 pseudo-simulation, arXiv 2506.04218 Primary}

What to look at instead

NAVSIM v2 now fills the gap with pseudo-simulation on real data.^{[2][2] NAVSIM v2 pseudo-simulation, arXiv 2506.04218 Primary}

• NAVSIM v2 Status:Still useful Yes. It is the main public driving benchmark, and its hard split stays at 34–37.

Where it appears in the report

Chapter 7, Multimodal, voice and embodied

• Figure 1. Tests Beyond Text Fill Up in Months, Too
• Driving moved to pseudo-simulation
• Table 14. Driving evaluation moved from open-loop metrics to closed-loop simulation, and then to pseudo-simulation on real data in “Go deeper: Autonomous driving benchmarks”