Definition
This happens when an AI model scores at or near the maximum possible on a test designed to measure a risky capability, meaning the test can no longer tell us how much further that capability has actually advanced. For sensitive areas like dual-use biology knowledge, a saturated benchmark leaves evaluators unable to confirm whether newer models are becoming more dangerous or just as capable as before.
Why it matters
Regulators and safety teams rely on these benchmarks as an early-warning system; once saturated, that warning system effectively goes dark, so decision-makers need to know when their risk metrics have stopped being informative.