Anthropic CEO Dario Amodei admits he does not know how US used Claude in strikes on girls school in Iran

Anthropic CEO Dario Amodei has accepted that the AI company does not know how the US military used its Claude AI models during a strike on an Iranian school in February that killed 175 people, largely schoolgirls. However, Amodei insists that this terrible incident only highlights the need for strict AI use policies.

In Short

• Amodei said Anthropic lacks access to how Claude was deployed in strike on Iranian school
• He called the school bombing terrible, emphasised on need for AI use principles
• US strike on the school said to have killed 175 people, mostly school girls

Earlier this year, the US Pentagon announced that it was going to stop using Claude AI models after a fallout with Anthropic over unrestricted AI use. While the US military wanted to use AI for “all lawful purposes,” the AI startup wanted to restrict the use in certain cases. But in February, when the US started strikes in Iran, Claude was still being used by the Pentagon, and is said to have played a role in a strike on a school that killed 175 people. But Anthropic’s CEO, Dario Amodei, says that he has no idea how his company’s AI tools were used in this strike.

“Look, we don’t have access to, we don’t know exactly how these models were used,” Amodei told Bloomberg. The Anthropic chief claimed that the strike was “a really terrible thing to happen.”

The Pentagon has not claimed responsibility for the incident, but reports indicate that US forces were likely behind the bombing. As per reports, the US military may have launched strikes on the school based on outdated intel that wrongly identified the area as still being a military site. Though it is unclear how the AI tools may have played a role in the incident.

School strike shows why AI principles are important, says Dario Amodei

One thing that Dario did emphasise upon was that it is likely that a human made the final call regarding the strike, something that was one of the demands from Anthropic over the use of its models by the military. He explained, “The principle that we have established, and I think the principle that was obeyed here, is that a human makes the final decision.”

According to the Anthropic CEO, the terrible incident only shows why such principles are necessary as militaries start deploying AI tools in combat. He added, “I don’t know what role Claude or any other AI had, but if this isn’t an illustration why that principle is so important, I don’t know what is.”

Amodei also said that the use case, if Claude was used in the operation, did not violate the company’s policies. According to the report, he argued that military leaders make mistakes “even at the best of times”. He also defended the broader use of AI in conflict, saying, “We don’t want a world where China and Russia can build, can analyse all the intelligence with AI, can use AI for attacking Taiwan and Ukraine, and we can’t defend them.”

Do note that Anthropic’s AI models are said to be phased out by the US Department of Defense soon. The AI startup was labelled a “supply chain risk” after the Pentagon refused two conditions – to not use AI for domestic mass surveillance, and to not develop autonomous weapons. The company has gone to court over this label. OpenAI is said to fully replace Anthropic’s models in the US military.

However, recent reports indicate that US agencies like the National Security Agency (NSA) are using Claude Mythos to potentially plan global cyberattacks.

- Ends