Benchmark checker Safety and security

Is Virology Capabilities Test (VCT) still meaningful?

Still useful Checked against the report, 5 November 2026

Yes, as a warning signal. A high paper score may not mean real-world uplift.

Why still useful: The chapter lists it as active: o3 scored 43.8% in 2025, above 94% of experts.^{[1][1] Virology Capabilities Test, arXiv 2504.16137 Primary}

What “Still useful” means

Known problems

The authors of a wet-lab trial flag a gap between paper benchmarks and the physical lab.^{[2][2] Wet-lab uplift trial, arXiv 2602.16703 Primary}
SecureBio co-developed xAI's bio/chem benchmark and licenses VCT to several labs.^{[3][3] xAI, Risk Management Framework (20 Aug 2025) Primary}^{[4][4] SecureBio, AI: 2025 in Review Vendor}

What to look at instead

The report names no direct successor.

Where it appears in the report

Chapter 1, Benchmarks

Table 18. Dangerous-capability benchmarks for CBRN, cyber and autonomy, November 2026 in “Go deeper: Dangerous capabilities: CBRN, cyber and autonomy (7)”

Chapter 4, Market and companies

Table 9. Safety evaluators in “Go deeper: Expert data, RL environment and safety evaluator tables, with detail”

Chapter 5, Institutions and regulation

Labs grade themselves under their own frameworks, with outsiders in deeper roles

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Virology Capabilities Test, arXiv 2504.16137. It compares models with expert virologists. Primary arxiv.org Chapter 1, source 142
[2]
Wet-lab uplift trial, arXiv 2602.16703. It tests LLM uplift in a viral genetics workflow. Primary arxiv.org Chapter 1, source 148
[3]
xAI, Risk Management Framework (20 Aug 2025). It lists xAI's benchmarks and thresholds. Primary data.x.ai Chapter 5, source 98
[4]
SecureBio, AI: 2025 in Review. It lists its benchmarks, lab work and government contracts. Vendor securebio.pages.dev Chapter 5, source 43