What happened
Based on 37 semi-structured expert interviews with frontier-AI governance specialists (from MATS, the Oxford Martin AI Governance Initiative and Mila), the authors reconstruct the implicit case for 'evaluation-based governance' — using AI capability evaluations as the empirical basis for risk classification and mitigation. They identify three pathways by which evaluations could reduce risk (making societal-level risks concrete to decision-makers and the public, cited in 18 of 37 interviews; serving as a basis for governance commitments and regulation, 18; and incentivising a better risk culture within AI companies, 16) and four principal challenges (diminishing informativeness of evaluations as capabilities improve, 21; measurement limitations inherent to the evaluation paradigm, 18; no guarantee evaluation results will be acted on, 16; and the degree of control and discretion AI companies retain over evaluation design, disclosure and response, 12). Four participants said evaluation-based governance could prove counterproductive by functioning as a 'permission structure' for continued capability improvement. Published at AIES 2026 (peer-reviewed, not a preprint-only note), the paper critiques the very paradigm now codified in the EU AI Act, California SB-53 and the Frontier AI Safety Commitments.
Why it matters
This is the strongest critical analysis this week of the evaluation regimes regulators and frontier labs are now building on; it names concrete failure modes an executive should weigh before treating benchmark results as a governance guarantee.
Action needed
Brief your risk and compliance leads on evaluation saturation and developer control of evaluation design, and press model vendors on independent verification of their capability claims.