UN calls for urgent action to rein in AI risks

The United Nations rights chief has called for countries and "frontier" artificial intelligence companies to act urgently to rein in a technology he warned posed "unprecedented risks".

In a letter sent to countries and AI developers and shared with the media, Volker Türk cautioned that the world was "on the cusp of irreversible change, affecting not just us but generations of humanity to come".

"The current race towards ever more powerful AI is a step change towards greater existential risks to every aspect of our lives," Mr Türk said.

He sent the letter at a time when worries over the risks of "superintelligent" computer systems have been on full display, after a researcher publicly resigned from leading AI firm Anthropic over fears that the technology could escape human control.

Another Anthropic employee, who did not resign, stated publicly that "we really do earnestly believe AI could kill all humans", and that he thought the chances were greater than "10% within the next decade".

Amid the outcry, Anthropic CEO Dario Amodei called Saturday for AI companies to "pace the frontier", or coordinate a slowdown in the technology's development, with his chief competitors, xAI's Elon Musk and OpenAI's Sam Altman, publicly supporting him.

"In today's world, we are almost entirely reliant on a small number of powerful companies to make the right choices on Al development for all of humanity," Mr Türk said.

"I call on States and companies to mobilise urgently through multilateral fora to chart a path for action to govern advanced, frontier AI models, grounded in international human rights law," he added.

Meanwhile, US President Donald Trump has downplayed the need for his administration to slow the development of artificial intelligence, saying he worried about ceding America's edge over China.

While acknowledging the need for some regulation, Mr Trump provided no specifics about potential rules and attributed warnings about the technology moving too quickly to "negative forces" he did not identify, though industry pioneers have raised such concerns.

"We can put guardrails, we can do this and that, but I think you have a lot of negative forces that are bringing it up that … shouldn't be bringing it up, and they're bringing up things that won't happen," Mr Trump told reporters during a trip to Ireland.

"But whoever wins with AI wins."

Chinese President Xi Jinping is due to visit the US later in September to meet Mr Trump, and AI will probably be among the items they discuss.

Kevin Hassett, director of the National Economic Council at the White House, said he expects meetings will be held this week involving administration officials who oversee cyber and science and technology policy to discuss the next steps.

Mr Trump also could meet members of Congress, some of whom have also been sounding alarms about the fast pace of AI development.

"I think the president wants to hear from them as well," Mr Hassett said.

US House speaker Mike Johnson acknowledged the need for "some guardrails, some safety measures" so AI "doesn’t run away".

But, like Mr Trump, he also wants to ensure the US does not "lose our edge in the global competition on this with China because that would be a national security threat".

Additional reporting: Press Association