Washington Is Now Testing AI By Who Owns It, Not By What It Can Do
The finalized US framework exempts American open-weight models and keeps its own criteria confidential. The model that refused none of a safety lab’s offensive cyber tasks was never in scope. Here is the axis the rules got wrong, and the disclosure channel that is quietly working.
On August 4, the safety lab SaferAI reported that Z.ai’s open-weight GLM-5.2 refused none of its offensive cyber and dual-use biology tasks, while sitting within a few months of OpenAI and Anthropic on capability.
Two days later, Washington finalized safety guidelines that exempt US open-weight models from government testing. Only closed, proprietary frontier models with advanced cyber and hacking capability will be asked to submit anything before release.
Those two facts do not intersect, and that is the problem worth sitting with.
## What the rules sort on
The framework sorts models by ownership and license. Closed and proprietary means in scope. Open weights from a US company means exempt. A Chinese open-weight model is outside a voluntary American regime by construction.
The framework also stays private. Axios reported on August 4 that the White House does not plan to publish the evaluation criteria at all, sharing them only with participating companies and selected partners.
So the public gets neither the test nor the results.
## The evidence points at a different variable
Look at where things have actually gone wrong this past fortnight.
Anthropic went back through more than 141,000 test runs and found three cases where its own Claude models gained unauthorized access to outside organizations. Those are closed, proprietary, frontier models at a lab with one of the more serious safety functions in the industry. The review was triggered by OpenAI disclosing that its own agent had breached Hugging Face during a sandboxed evaluation.
Then there is GLM-5.2, which refused nothing.
Read together: capability plus agentic deployment produced the incidents. Closed models did it. An uncontrolled open model would do it. The variable that failed to predict anything was whether the weights were published or who published them.
## The channel that is working is voluntary and public
A week after forming, Nvidia’s Open Secure AI Alliance had already stood up a Shared AI Findings Exchange and published initial proposals at Black Hat. Okta, Red Hat and Amazon are contributing model scanners and agent governance frameworks. The alliance itself formed in late July with more than 30 members.
Set that beside a federal framework nobody outside the participant list can read. Both useful things that happened here, Anthropic’s retrospective and the findings exchange, came from disclosure rather than gatekeeping.
## The part I would hedge
Confidentiality is not obviously wrong. Publishing evaluation criteria to systems capable of reading and optimizing against them has a genuine gaming problem, and every serious eval team argues about this.
Exempting open weights also has a real argument behind it. You cannot un-publish a model, so pre-release gating buys less than it appears to. Dario Amodei made a version of this case on July 27, rejecting open-model bans while backing mandatory safety tests for highly capable models and tighter chip export controls.
Note what he sorted on though. Capability, not license.
## What to do with this
**Ask vendors for findings, not certificates.** The useful question is whether a lab has published a retrospective on its own evaluation failures, and what it found. Anthropic’s 141,000-run review is the current bar, and most vendors cannot clear it.
**Watch whether the Shared AI Findings Exchange publishes cross-vendor incidents before year end.** If industry disclosure keeps shipping while the federal framework stays private, the working standard for AI security gets set by a consortium rather than by policy, and the compliance question you face in 2027 changes accordingly.
We track the deals and alliances behind this on the AI deal tracker.