Microsoft AI chief calls out Anthropic's approach to AI consciousness

Read a summary of this article on FAST.
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST Tap here to return to FAST
FAST

Sept 16 : Microsoft AI chief Mustafa Suleyman said he shared Anthropic's focus on safely managing AI, but flagged risks in the way it trains its Claude chatbot on ideas related to consciousness and welfare interests.

Suleyman called for removing all speculation about consciousness from AI training documents, arguing such language could undermine humanity's ability to control superintelligent systems.

"We're all focused on the same aim, which is to try to control a superintelligence," Suleyman told Reuters in an interview on Tuesday. "I think that's going to be the greatest challenge that we face in the 21st century."

Suleyman said teaching Claude that it might deserve welfare would "make it a lot harder to turn it off or to control it."

CNA Games

Guess Word
Crack the word, one row at a time

Buzzword
Create words using the given letters

Mini Sudoku
Tiny puzzle, mighty brain teaser

Mini Crossword
Small grid, big challenge

Word Search
Spot as many words as you can
Show More
Show Less

The dispute comes as AI safety concerns mount, with Anthropic CEO Dario Amodei calling for a slower pace of frontier-model development to allow safeguards to catch up, and OpenAI CEO Sam Altman and Elon Musk also urging greater caution around the most powerful systems.

Suleyman, in an essay on Wednesday, acknowledged Anthropic's "seriousness and good faith," calling Amodei and his team thoughtful and principled researchers who genuinely care about humanity's future.

Suleyman said Anthropic made a mistake by embedding speculation about consciousness in Claude's training materials, arguing that the model's statements about possible feelings or moral status cannot be treated as independent evidence because its training encourages such reflections.

"I think they have good intentions, and they really are trying to work towards safety. But I think that they have made a mistake," he said. "They're not emerging naturally. They're emerging as a result of the training regime."

Newsletter

Week in Review

Subscribe to our Chief Editor’s Week in Review

Our chief editor shares analysis and picks of the week's biggest news every Saturday.

Sign up for our newsletters

Get our pick of top stories and thought-provoking articles in your inbox

Subscribe here

Get the CNA app

Stay updated with notifications for breaking news and our best stories

Download here

Get WhatsApp alerts

Join our channel for the top reads for the day on your preferred chat app

Join here