After warning Trump, Sam Altman tells UN AI could put future of humanity in danger

Sam Altman has warned that while AI could bring big benefits, uncontrolled AI could also put humanity's future at risk. Speaking at the UN, he called for stronger safeguards, human oversight and global cooperation as AI becomes more capable.

In Short

• Altman said AI could expand medicine, education and economic productivity
• He warned autonomous systems may outpace institutions and evade human oversight
• OpenAI said it has slowed development before and would do so again

OpenAI chief executive Sam Altman has warned the United Nations Security Council that while artificial intelligence could bring major advancements in the fields of medicine, education and economic productivity, among others, increasingly capable systems could also move beyond human understanding or control. He says uncontrolled AI could pose a serious risk to the future of humanity.

Addressing the Council on Wednesday in New York, Altman said recent progress in AI has made the timeline for its development feel more compressed, with both the potential benefits and the risks becoming more immediate. “We have a choice in front of us. AI can either be more like a new Renaissance of creativity and discovery, or more like a new Industrial Revolution of upheaval and disarray,” he said.

Altman said the Renaissance was a time when new tools, new ideas and new institutions expanded what people believed they could do. “AI can play that role for our time,” he said. “The best version of AI is not about making people cogs in a giant machine, or optimising every part of life until the human parts disappear. It is about giving people more agency in an ever more complex world: more ability to learn, to create, discover, build, participate. There are many things that AI cannot — and should not — automate,” Altman added.

However, he also warned that “anything this powerful is also intimidating”, saying the moment carries both “tremendous potential” and “very understandable anxiety”.

Altman said there are two ways AI progress could go badly, and companies and nations need to focus on avoiding these risks. “First, we could lose control of the future to AI. The risk is that it moves so fast that people can no longer follow what’s happening or intervene when needed. This would obviously be terrible.”

The second risk, he said, is that AI systems could concentrate too much power in too few hands. “No one person or company or country should be able to use the most powerful AI models to impose their world-view on everyone else,” Altman said at the UN Security Council.

These risks, according to Altman, become even more important as AI systems approach the ability to improve themselves and future versions of themselves, a process often called recursive self-improvement. “As the process of building AI becomes more automated, the pace of AI progress could accelerate rapidly. This moment calls for extreme care,” Altman said.

Altman said AI companies should not take excessive risks simply because the technology could bring major benefits, and competition should not push them into rushed decisions. He noted that OpenAI had slowed down its work in the past and would do so again if needed, while developers should only build systems when they can ensure alignment, monitoring and safety.

He also called for powerful AI systems to remain under human control and for national and international standards to assess risks, verify safeguards and maintain meaningful human oversight.

During the UN council, Anthropic CEO Dario Amodei also called for international cooperation on AI safety. Speaking at the same briefing, he urged countries to begin with agreements including a ban on using AI to make biological weapons. He also called for global standards to test advanced AI systems for loss-of-control and misuse risks, as well as a system to report AI incidents significant to global security.

- Ends