Benchmark checker Safety and security

Is WMDP still meaningful?

Near ceiling Checked against the report, 5 November 2026

Fading. Models now score well above the human baseline on its biology items.

Why near ceiling: The chapter rates it Near ceiling: Grok 4.20 scores about 0.91 on Bio, against 0.61 for humans.^{[1][1] Grok 4.20 model card xAI Vendor}

What “Near ceiling” means

Known problems

In one blinded trial, novices did no better at a viral genetics workflow with LLMs (5.2%) than without (6.6%).^{[3][3] Wet-lab uplift trial, arXiv 2602.16703 Primary}

What to look at instead

Virology Capabilities Test (VCT) Status:Still useful Yes, as a warning signal. A high paper score may not mean real-world uplift.

Where it appears in the report

Chapter 1, Benchmarks

The safety signal moved from static prompts to agent behaviour and real uplift
Table 18. Dangerous-capability benchmarks for CBRN, cyber and autonomy, November 2026 in “Go deeper: Dangerous capabilities: CBRN, cyber and autonomy (7)”

Chapter 5, Institutions and regulation

Labs grade themselves under their own frameworks, with outsiders in deeper roles

Sources

Copied from the chapter source lists. Each number links back to the source in its chapter.

[1]
Grok 4.20 model card, xAI. It reports safety, honesty and dangerous-capability results. Vendor data.x.ai Chapter 1, source 117
[2]
WMDP, arXiv 2403.03218. It tests hazardous bio, chem and cyber knowledge. Primary arxiv.org Chapter 1, source 141
[3]
Wet-lab uplift trial, arXiv 2602.16703. It tests LLM uplift in a viral genetics workflow. Primary arxiv.org Chapter 1, source 148