AI development needs 'extreme care', OpenAI boss tells UN

The current moment in AI development "calls for extreme care," OpenAI CEO Sam Altman has told the UN Security Council, noting that the technology's advancement could accelerate rapidly.

"The industry must not accept too much technological risk just because the benefits are too great and important to slow down," Mr Altman told the United Nation's top body.

He was among tech industry leaders briefing an emergency meeting of the 15-member council, following weeks of global headlines about the risks of uncontrollable artificial intelligence and calls for a slowdown.

Besides Mr Altman, Dario Amodei of Anthropic also addressed the session. Clement Delangue of Hugging Face is also expected to speak.

Mr Altman said his company would try its best to walk a "middle path".

While "we don't want to fall into the trap of blind optimism," he cautioned against falling into "doomerism" too.

Recently, OpenAI called for the United States to lead a global effort on establishing standards for artificial intelligence, noting that the US industry is at the technical frontier.

Mr Amodei earlier told the council that if managed poorly: "I even believe that AI could be a risk to humanity as a whole."

Governments have grappled with how to oversee AI systems that are advancing faster than regulators can keep up while the United States and China — the two dominant players in AI development — remain divided over whether the technology needs global guardrails or should be left largely to market forces.

"If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone. ‌They must be shaped through democratic processes and ⁠by governments accountable to the people they serve," Mr Altman told the council. "At the international level, this will require cooperation."

"In our history, there have been times where countries who compete and don't always like each other very much still come together for shared interests and the collective good in the face of a powerful new technology. We believe this must be one of those times," he said.

Yoshua Bengio, a Canadian considered one of the "Godfathers of AI" and co-chair of the Independent International Scientific Panel — created by the UN ‌General Assembly last year — told the Security Council: "The dangers are real and imminent."

"This council faces an unprecedented threat, one that none of its members would choose, that none can contain alone, and that does not respect the borders we ⁠defend," he said.

Council members, including leading AI rivals the United States and China, were also set to speak. French Foreign Minister Jean-Noël Barrot chaired ‌the meeting.

Trump wants to encourage AI, not rein it in

Ahead of a meeting between US President Donald Trump ⁠and Chinese leader Xi ‌Jinping in Washington tomorrow, the US and China have discussed setting up a notification system for common goals and common threats that would cover AI-related incidents that rise to a national security level.

Mr Trump told world leaders at the UN General Assembly on Tuesday that he "rejects any attempt to construct a globalist scheme to control" AI, describing the technology as amazing, but noted ⁠that the US would be careful.

"We're leading now over China by a lot, and everyone else, and we're going to keep it that way," Mr Trump said. "I'm not ⁠going to stifle growth of something that will be bigger than the Industrial Revolution."

"We're going to encourage it, not rein it in. We're going to watch it closely through the Department of Justice," he said.

UN Secretary-General Antonio Guterres, who has long pushed for global governance of AI, has warned that rapidly advancing artificial intelligence poses risks that cannot be ignored, putting him at odds with Mr Trump.

The Security Council met for the first time on the risks of AI in 2023, when China said the technology should not become a "runaway horse" and the United States warned against its use to censor or repress ‌people. Then-US Secretary of State Antony Blinken chaired a Security Council meeting on AI in 2024.