What happened
RAND researchers propose a proportional evaluation (PE) framework tailored to open-weight AI models, which introduce distinct risk factors not addressed by evaluation practices designed for closed-weight deployments. The authors systematically reviewed evaluation practices for 37 families of open-weight models released between 2025 and April 2026, finding that only one fulfills all four PE criteria (PE1-4) and most do not fulfill any. The framework addresses the gap between current evaluation norms—which assume controlled deployment—and the realities of open-weight models that can be fine-tuned, quantized, and deployed without oversight.
Why it matters
Open-weight models are proliferating (37 families in ~16 months) but lack evaluation standards proportional to their unique risks. Organizations building on or deploying open-weight models face an evaluation gap: existing benchmarks do not assess post-release risks like fine-tuning for harmful tasks or deployment at scale by non-expert actors. This framework provides a structured basis for policy and procurement decisions.
Action needed
If your organization uses or plans to use open-weight models, compare the models against RAND's PE1-4 criteria to identify evaluation gaps. Discuss with your AI governance team whether your vendor selection and risk assessment processes account for post-deployment risks specific to open-weight architectures.