Dario Amodei says Anthropic has made many mistakes in race to the top, will do better

Anthropic CEO Dario Amodei has defended the company's AI safety record after criticism from advocate Nathan Calvin over its priorities. Amodei said Anthropic has made commercially costly decisions to reduce AI risks, while Calvin questioned whether safety is keeping pace with the company's push to build more powerful AI models.

In Short

• Anthropic CEO Dario Amodei defends the company’s AI safety record after criticism
• Nathan Calvin questions whether Anthropic is prioritising AI development over safety
• Amodei says the company has made commercially costly decisions to reduce AI risks

Anthropic CEO Dario Amodei has defended the company’s approach to AI safety after several people raised concerns about its recently proposed AI safety bill. A few days ago, Anthropic proposed a federal AI safety bill and reportedly shared a draft with US lawmakers. Soon after the news became public, it drew a fierce reaction from civil society groups, tech policy watchdogs and lawmakers. Among the many objections, one concerns the threshold for government intervention, which requires 50 deaths or $1 billion in damages before regulators can act. Critics argue this threshold is far too high.

Nathan Calvin, chief strategy officer at AI policy and safety advocacy group Encode AI raised concerns on X, formerly known as Twitter, questioning whether the company was putting its ambition to build increasingly powerful AI models ahead of its safety commitments.

In a lengthy response to Calvin’s criticism, Amodei said, “I think it is a general truth that any decision, on any topic, can always be portrayed as a nefarious scheme.” He argued that Anthropic has repeatedly taken decisions that came at a commercial cost to the company but helped reduce the risks associated with advanced AI. He also acknowledged that the company had made mistakes and that some of Calvin’s criticisms were valid.

Calvin argued that Anthropic’s actions over the past year raised questions about its commitment to AI safety. In his post, Calvin said the company’s actions appeared more consistent with a focus on winning the race to develop artificial intelligence than with prioritising safety. He pointed to Anthropic’s plans to develop custom chips, large investments in computing infrastructure, increased use of AI models in internal development and changes to its Responsible Scaling Policy.

Calvin also criticised the company’s proposed federal AI safety bill, arguing that some provisions could weaken existing state-level protections and make it harder for the government to stop the development or deployment of potentially dangerous AI models.

He acknowledged that Anthropic had also published important safety research and worked with external organisations. However, he questioned whether these efforts were keeping pace with the company’s push to develop more capable AI systems.

“Why do the mistakes systematically tend to guard and favour Anthropic’s interests?” Calvin asked, questioning whether the company’s commercial incentives could influence its approach to safety.

Dario Amodei defended Anthropic

Responding to Calvin, Amodei said Anthropic’s strategy has long been to pursue what he calls a “race to the top”. The idea is that by developing advanced AI responsibly, the company can encourage other AI developers to follow stronger safety practices.

Amodei argued that being at the frontier of AI development gives Anthropic greater influence over the industry. He said he remained proud of the company’s record, despite acknowledging that it had made mistakes.

“To be fair, those that involved holding back or restricting access to frontier capabilities also had a broader cost,” Amodei wrote, acknowledging that delaying access to AI models could also slow potential benefits, including progress in biology.

To support his argument, Amodei listed several decisions that he said had carried high commercial costs. These included supporting California’s SB 53 AI safety law, maintaining strict safeguards against the misuse of AI for biological threats and delaying the wider release of advanced models until stronger cybersecurity protections were in place.

He also said Anthropic had advocated for government powers to block dangerous AI model releases, supported export controls on advanced chips to China and restricted the sale of its AI services to companies controlled by the Chinese government.

According to Amodei, these decisions demonstrate that the company has been willing to sacrifice business opportunities to address safety and national security risks.

Anthropic’s proposed AI safety bill faces scrutiny

The disagreement comes amid a wider debate over Anthropic’s attempt to shape federal AI regulation in the US. According to a report by Politico, Anthropic shared a draft bill with some congressional offices that would establish requirements for independent evaluations, safety frameworks, transparency reports and the reporting of critical safety incidents.

However, critics argued that the proposal could restrict the circumstances in which the government could halt an AI model’s development or release. They also raised concerns about provisions that could override certain state AI safety laws.

Anthropic’s public policy team said the draft was intended to support a strong national AI safety framework. The company also acknowledged that parts of the proposal could be improved and said it planned to continue discussions with policymakers.

The debate highlights a difficult question facing the AI industry: can companies continue to develop increasingly powerful models while ensuring that safety measures keep pace? While Amodei argues that remaining at the technological frontier gives Anthropic greater influence to promote responsible development, Calvin believes the company’s actions deserve continued scrutiny from people outside the industry.

- Ends