What happened
This preprint (arXiv 2611.08134, from Oxford, the Centre for AI Security and Access, and the University of the Witwatersrand) introduces the PAAI framework — Power, Access, Adequacy, Independence — for analysing who controls the material and institutional resources on which rigorous AI evaluation depends. The authors apply it to twelve African AI evaluation projects (2022–2025) and to the documented roles of all ten member institutes in the three published joint testing exercises of the international network of AI Safety Institutes. They identify three recurring configurations — extractive collaboration, dependent innovation, and constrained independence — and conclude that "design authority without infrastructure authority never yielded more than partial independence": African and smaller-member contributors supplied data, language expertise and task design while external institutions retained authority over evaluation frameworks, methods and model execution. The paper's central claim is that participation in an evaluation does not by itself confer evaluative authority; credible governance requires access to come with control over priorities, methods, publication and continuity. Preprint, not peer-reviewed.
Why it matters
This is directly relevant to any board or policy lead who relies on third-party AI evaluations, audits or red-team findings — it gives a concrete test for whether those assessments carry real authority or merely represent participation, which matters as regulators and enterprises increasingly lean on independent evaluation as a governance lever.
Action needed
Map the PAAI dimensions (Power, Access, Adequacy, Independence) against the independent evaluations and audits your organization commissions or relies on, and require evaluators to demonstrate control over methods and publication — not just access to the model.