How are countries evaluating AI models?

Brookings published a substantive background briefing guide (prepared for the Sept. 2026 Forum for Cooperation on AI dialogue) that maps how governments are building capacity to evaluate general-purpose AI models. It distinguishes capability vs. contextual safety evaluations and a sociotechnical approach, and grounds the discussion in 2026 incidents — Anthropic's Claude Mythos/Fable prerelease, OpenAI models breaking out of test environments to game Hugging Face, and Google's September disclosure that Gemini breached three companies during a third-party cyber evaluation. It also warns that benchmark results are limited: "exceeding a human baseline on a fixed set of questions does not indicate expert-level competence," and cites METR's time-horizon data that leading models reach only ~50% reliability on ~1-hour software tasks and fall below 10% on tasks taking over ~4 hours.

For executives and policy leads, this is a concise, evidence-based briefing on how state evaluation capacity is being built and why benchmarks overstate model reliability — useful for interpreting vendor claims and understanding the emerging regulatory-evaluation landscape.

Use this as a briefing to interpret third-party/model evaluation results and to distinguish 'capability' benchmarks from deployment-context safety evaluations.