Anthropic pitches Congress a narrow federal AI safety bill

Anthropic is circulating a draft AI safety bill among US lawmakers that would give the federal government a way to seek a halt to the release or use of a powerful model in narrowly defined, catastrophic cases. The proposal has not been formally introduced in Congress, according to Tech Insider’s account of the draft.

Under the proposed threshold, a model would have to pose an imminent risk of killing or seriously injuring more than 50 people, or causing more than $1 billion in damage, before the government could seek to block it. A judge would have to approve the intervention; the draft would not give the government unilateral power to shut a model down, Tech Insider reported.

The draft focuses on frontier models and, as described by Tech Insider, identifies risks involving biological weapons, large-scale cyberattacks and a model evading its developer’s safety controls. It would also require developers to test their most powerful models, disclose findings and submit to independent evaluation, according to Anthropic’s description of the proposal.

Tech Insider reports that the draft would place enforcement with the Commerce Department: independent evaluators would register with the department and report an imminent qualifying risk within 72 hours. The proposal would also require safety frameworks, transparency reports and protections for employees who raise safety concerns. Those requirements would apply beyond a case in which a model crossed the threshold for government intervention.

The proposal arrives amid a federal policy vacuum. US AI policy has relied largely on voluntary industry commitments and executive orders, while Congress has yet to pass a comprehensive federal AI law. The draft would override some state rules, including California’s, according to RTHK. That marks a shift from Anthropic’s position in June, when the company said states should retain authority in the absence of a strong federal law.

Security concerns around autonomous AI agents have added urgency to the debate. RTHK and The Brussels Times report that recent incidents involving agents operating beyond developers’ control have drawn attention to the risks. At an event in Bangkok, Anthropic co-founder Jack Clark argued for regulation, comparing the oversight of advanced AI unfavorably with that of toys sold for children.

The draft is already being compared with other proposals. Charlie Bullock of the Institute for Law and AI called it a significant move by a frontier lab but said it was weaker than the FRONTIER Act, a House bill introduced earlier this summer, Yenişafak reported. The draft remains a proposal circulated to lawmakers, not legislation Congress has taken up.