Anthropic says AI labs need coordinated plan to halt development if risks rise
· CNA · JoinRead a summary of this article on FAST.
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST Tap here to return to FAST
FAST
June 4 : Anthropic said on Thursday frontier AI developers should establish a coordinated, verifiable way to slow down or temporarily pause development if advanced systems begin improving themselves faster than society can manage the risks.
AI that can build itself would be a major development in the history of technology, but "full recursive self-improvement also might increase the risks of humans losing control over AI systems," the AI startup said.
"If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important."
As an example, Anthropic said that as of May, more than 80 per cent of the code merged into its codebase was authored by Claude.
CNA Games
Guess Word
Crack the word, one row at a time
Buzzword
Create words using the given letters
Mini Sudoku
Tiny puzzle, mighty brain teaser
Mini Crossword
Small grid, big challenge
Word Search
Spot as many words as you can
Show More
Show Less
It would be "good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology," the company said.
However, it cautioned that unilateral or poorly coordinated slowdowns could backfire if less cautious actors continue advancing, potentially reducing overall safety.
It highlighted that a meaningful pause would require agreement among "multiple well-resourced labs" operating at the technological frontier, as well as rules on what conditions would trigger or lift such a pause and who would oversee it.
A unilateral pause by a single company would be easier to implement, Anthropic added, but would have limited impact, primarily shifting leadership rather than fostering broader global deliberation.
Its research arm, Anthropic Institute, plans to study and help build systems that would be necessary to support a slowdown.
In the coming months, Anthropic plans to convene discussions involving policymakers, researchers, civil society groups and other AI firms to examine key questions.
These questions include how to manage AI-related risks such as recursive self-improvement and how to improve mechanisms for coordination.
Last month, Anthropic concluded a fundraising round that valued the company at $965 billion and confidentially filed for a U.S. initial public offering on Monday.
Newsletter
Week in Review
Subscribe to our Chief Editor’s Week in Review
Our chief editor shares analysis and picks of the week's biggest news every Saturday.
Sign up for our newsletters
Get our pick of top stories and thought-provoking articles in your inbox
Get the CNA app
Stay updated with notifications for breaking news and our best stories
Get WhatsApp alerts
Join our channel for the top reads for the day on your preferred chat app