Anthropic’s draft AI bill would let courts halt models in narrowly defined emergencies

Anthropic has confirmed it is circulating draft legislation among U.S. lawmakers that would let the federal government seek to block the release or use of powerful AI models in narrowly defined catastrophic-risk cases. The proposal has not been formally introduced in Congress, and its future is uncertain.

Under the reported draft, the Commerce Department could ask a federal judge to suspend a model’s development, deployment or use if there were an imminent risk of killing or seriously injuring more than 50 people, or of causing more than $1 billion in damage. Covered scenarios include an AI system assisting with a biological weapon, enabling a serious cyberattack or escaping its developer’s controls. The court requirement would set the proposal apart from a system in which federal officials could unilaterally shut down a model.

The bill would also give independent evaluators a formal role. They would register with the Commerce Department and periodically assess frontier companies’ testing and safety measures. Companies could pay for reviews, but not in a way that depended on favorable findings, and evaluators who identified a qualifying imminent risk would have 72 hours after completing an assessment to notify the department, according to the reported draft.

Anthropic says the legislation would require frontier developers to test their most powerful models, publicly disclose their findings, undergo independent evaluation and maintain robust security programs. Its broader framework targets the largest developers through thresholds for training compute and for company revenue or research spending, rather than applying the same requirements to every AI startup.

The proposal arrives at a time when the U.S. has relied mainly on voluntary industry pledges and executive orders, and Congress has yet to pass a federal AI law. It would also override some state rules, including laws in California, according to CGTN. That marks a notable shift from Anthropic’s June position that states should retain authority absent strong federal legislation.

The regulatory debate is unfolding alongside questions about who should be liable when AI systems cause harm. A Reuters report described negligence as one possible route under existing law, while legal experts cited in the report discussed strict liability and shifting the burden of proof as alternatives. Those questions are separate from Anthropic’s draft, which is aimed at preventive government intervention rather than compensation after harm occurs.

Initial reaction has not been uniformly favorable. Charlie Bullock of the Institute for Law & AI called the proposal reasonable and said it went further than other frontier labs, but also characterized it as weaker than the House’s competing FRONTIER Act, according to AFP’s reporting. With the draft still at the proposal stage, there is no guarantee lawmakers will take it up or adopt its terms.