Tech

Five AI labs now let the US government test their models before release. The arrangement is voluntary, has no legal basis, and is the closest thing America has to AI oversight.

In a bid to mitigate AI risks, five leading research labs have begun voluntarily submitting their models to US government testing before public release, effectively establishing a de facto oversight framework in the absence of formal regulations. This unprecedented arrangement, driven by the Mythos crisis, allows the government to assess AI systems' potential national security threats before they reach the public domain. The Commerce Department is now at the forefront of this informal AI governance effort. AI-assisted, human-reviewed.

Overview

Google, Microsoft, and xAI have joined OpenAI and Anthropic in voluntarily giving the US Commerce Department pre-release access to evaluate their AI models. The five companies now account for the vast majority of frontier AI development worldwide, and all five have agreed to let a single government office test their systems before deployment. The arrangement is voluntary, has no statutory basis, and gives the government no power to block a release. It is also the closest thing the United States has to an AI oversight system, and it was built in less than two years by an office with fewer than two hundred staff.

The office

The Center for AI Standards and Innovation sits within the Commerce Department’s National Institute of Standards and Technology. It was established under President Biden in 2023 as the AI Safety Institute, re-established under Trump with a new name and a reorientation toward standards and national security rather than safety research. The centre has completed more than 40 evaluations of AI models, including state-of-the-art systems that have never been released to the public. Developers frequently submit versions with safety guardrails stripped back so that evaluators can probe for national security-relevant capabilities: biological weapon synthesis pathways, cyberattack automation, and autonomous agent behaviours that could be difficult to control at scale.

Chris Fall now directs the centre, following the abrupt departure of Collin Burns, a former AI researcher at Anthropic who was chosen for the role but pushed out by the White House after four days. Burns had left Anthropic, given up valuable stock, and relocated across the country to take the government position. His removal, reportedly driven by his connection to a company the administration was actively fighting, illustrates the political complexity of building an oversight system for an industry where the evaluators and the evaluated come from the same talent pool.

The agreements

The new partnerships with Google, Microsoft, and xAI expand what had been a two-company arrangement into something closer to comprehensive frontier coverage. OpenAI and Anthropic have renegotiated their existing agreements to align with Trump’s AI Action Plan, which directs the centre to lead national security-related model assessments and positions it as part of a broader “evaluations ecosystem.”

The agreements are not contracts. They are voluntary commitments that the companies can withdraw from at any time. No statute requires pre-release evaluation. No regulation gives the centre authority to delay or block deployment. The entire system depends on the AI companies deciding, for their own strategic reasons, that giving the government early access is preferable to the alternative.

The alternative, from the companies’ perspective, is legislation. Several draft bills would give the centre permanent statutory authority, mandatory evaluation requirements, and the power to impose conditions on deployment. The Pentagon has already demonstrated willingness to blacklist AI companies that refuse

Similar Articles

More articles like this

Tech 1 min

The new AirPods Max 2 are already on sale for $40 off

Apple's AirPods Max 2, a true sequel to the original, has already seen a $40 price drop just over a month after its release, now available for $509 across all five colors at major retailers. This discount brings the premium over-ear headphones within striking distance of competitors like Bose and Sony, despite retaining the same 40-millimeter drivers as the original. The Max 2's new high dynamic range amplifier and lossless audio support via USB-C may justify the premium, but the price cut is a welcome development for consumers. AI-assisted, human-reviewed.

Tech 1 min

Five major publishers are suing Meta over Llama. They have evidence that the previous plaintiffs did not.

Five major publishers and a prominent author have launched a class-action lawsuit against Meta, alleging that the company pirated millions of their copyrighted works to train its Llama AI model, citing new evidence that previous plaintiffs lacked. The complaint specifically targets Meta's use of copyrighted materials in Llama's training data, which the plaintiffs claim has caused significant market harm. The lawsuit seeks damages and injunctive relief for the alleged copyright infringement. AI-assisted, human-reviewed.

Tech 1 min

Anthropic ships ten financial-services agents and pulls Moody’s inside Claude. The bank-software business is being rewritten.

Anthropic's Claude platform is rapidly expanding its presence in the financial services sector, with the deployment of ten pre-built agents and the integration of Moody's data into its library, covering 600 million companies. This strategic move is further solidified by the launch of a Moody's native app and the integration of an FIS-built AML investigator at BMO and Amalgamated Bank. The $1.5 billion joint venture with Wall Street is rewriting the bank-software business. AI-assisted, human-reviewed.

Tech 2 min

Orchid, the buzzy Tame Impala synth, is back in a gorgeous clear colorway

A limited-edition Arctic clear variant of the $699 Telepathic Instruments Orchid—co-designed with Tame Impala’s Kevin Parker—reintroduces the cult-favorite chord organ synth to the market on May 5, blending vintage harmonic simplicity with modern polyphonic patch recall to sidestep music-theory barriers. AI-assisted, human-reviewed.

Tech 1 min

OpenAI is reportedly launching a phone for ChatGPT

A custom smartphone from OpenAI is reportedly in development, with mass production slated for early 2027 and a customized MediaTek Dimensity 9600 chip at its core, featuring an enhanced image signal processor (ISP) with HDR capabilities. This marks a significant shift in OpenAI's hardware strategy, moving beyond rumored collaborations with Apple's design chief Jony Ive. The phone's release timeline is closely tied to the Dimensity 9600's launch this fall. AI-assisted, human-reviewed.

Tech 1 min

QuantWare lands €152m to build the world’s largest open-architecture quantum processor fab in Delft

Dutch quantum computing startup QuantWare secures a record-breaking €152 million Series B investment, marking the largest private round for a dedicated quantum-processor company and the largest ever raised by a Dutch deeptech firm. The funding will fuel the construction of the world's largest open-architecture quantum processor fab in Delft, backed by a high-profile syndicate of investors. This milestone underscores the growing momentum behind European quantum computing initiatives. AI-assisted, human-reviewed.