What happened
On 29 October 2026 MLCommons (consortium of 125+ members incl. Microsoft, Google, Qualcomm, NVIDIA, OpenAI, Anthropic) launched Candela, a benchmarking-as-a-service platform for AI risk and reliability, and shipped new versions of its consensus-built AILuminate benchmarks: Safety 1.2.1 (12 hazard categories) and Jailbreak 1.1 (11 categories, drawing on an updated jailbreak taxonomy spanning typos/lookalike characters up to context framing, deception and fragment assembly). Scoring uses a private ML safety classifier rather than an LLM-as-judge and item response theory, reporting results as percentiles against a fixed reference panel; official Safety 1.2.1 results for ~30 named open-weight models are already public. The AIRR working groups (80+ organizations) set what is tested; agentic-security, privacy, and psychosocial harm benchmarks are in progress.
Why it matters
This operationalises one of the few consensus, independently-scored measurement standards for AI risk and reliability. It gives deployers and procurement a citable, comparable signal (system cards, spec sheets, third-party assurance) instead of vendor self-claims, and it explicitly tests the system as deployed (system prompt, RAG, agent loop / tool harness) because safety scoring is not composable across configurations. Security and risk teams can use it to benchmark adversarial robustness and jailbreak resistance as part of an AI assurance program.
Action needed
Adopt AILuminate/Candela for benchmark-based risk assessment of AI systems and agentic deployments; track the coming agentic-security and privacy benchmark releases from the AIRR working group.