What happened
On 2026-08-27, MLCommons published a blog post ('The key to trustworthy AI evaluation is secrecy by design') describing its role as the reserved-benchmark provider in a pilot led by Google DeepMind, AVERI (AI Verification and Evaluation Research Institute), OpenMined, and Singapore's AI Safety Institute. The pilot ran a double-blind evaluation of Gemini 2.5 Flash-Lite inside a hardware-attested secure enclave (Google Cloud Confidential Space / NVIDIA H100 Confidential GPU, OpenMined PySyft), using never-before-used AILuminate AIRR 1.4 prompts, so that the evaluator never saw model weights and the developer never saw benchmark prompts. This is not itself a finalized standard but a methodology demonstration from a recognized AI-benchmarking consortium (MLCommons) explicitly framed as relevant to emerging third-party AI audit requirements (EU AI Act GPAI Code of Practice, Illinois SB 315 from 2028).
Why it matters
As jurisdictions move toward mandatory third-party frontier-model audits (EU AI Act GPAI CoP, Illinois SB 315), the lack of technical protocols reconciling evaluator confidentiality (benchmark integrity) with developer confidentiality (model weights) has been a structural gap. This pilot, endorsed by MLCommons as a benchmark steward, is an early technical reference point that regulators, auditors, and AI Safety Institutes may point to when defining acceptable third-party evaluation architectures.
Action needed
Watch — no adoption action required yet; AI governance/audit teams should track whether MLCommons or AVERI formalizes this secure-enclave protocol into a reusable specification or stewardship-program requirement.