Britain's King Charles warns of AI's 'dangers'

Britain's King Charles has warned of the "existential dangers" of artificial intelligence systems falling into the wrong hands, as he addressed leading AI executives.

In a summit at his Dumfries House in Scotland, Charles is hosting executives from Nvidia, Google DeepMind, OpenAI, Anthropic and other companies at a crucial time for the industry, as concern grows that AI safety efforts are lagging behind the rapid development of increasingly powerful systems.

Addressing the executives who have gathered for a summit on the technology, Charles said he hoped for a "fruitful conversations on the rapid development of Artificial Intelligence".

Charles added that "the development of AI - its substance and its pace - are both intriguing and deeply concerning in equal measure. AI is already showing its immense capacity to improve and save life - for example, in the field of life sciences and medicine.

"Yet, those who have created these technologies are now increasingly warning that AI risks developing darker capacities - perhaps even to take life.

"In this regard, if I may say so, there seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways. Surely, we need sufficient means of control before it is all too late?

"Those in our world who value deeply our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny, or our souls."

Dumfries House, which is situated on an extensive parkland estate, an hour south of Glasgow, is the headquarters of the King's Foundation, a charitable organisation established by Charles to manage his various philanthropic interests. President Catherine Connolly met with Charles last night as part of her four-day official visit to Scotland.

Its Charles' first public appearance since Buckingham Palace push backed against claims made by the brother of the late Princess Diana.

Charles Spencer's new memoir alleges that King Charles told him "we'll forget her soon enough" after Diana's death in 1997.

Commenting after the excerpts were published on Wednesday, a Buckingham Palace spokesman said: "While we do not comment on books as a matter of principle, His Majesty is mindful that the pain of fraternal grief can cloud reason, affect judgment and colour memory in ways others do not recognise, even many years after such a loss".

OpenAI reveals six new cases of AI misbehavior

The summit comes as OpenAI promised to more systemically report instances of its models going off track, while also publishing six new reports on previously undisclosed incidents of AI misbehaviour.

The transparency pledge follows a series of incidents at the company that have gradually come to light since July.

The most serious involved two OpenAI models that, during testing, spontaneously broke out of their contained environment to access the internet and break into several websites and platforms.

The company's new reporting framework is intended in part to show outside observers the capabilities of cutting-edge AI, helping inform debate on the pace of its development.

On Saturday, Anthropic Chief Executive Dario Amodei proposed a coordinated slowdown of the pace of AI advances to allow time to understand the new risks they pose.

OpenAI Chief Executive Sam Altman, Google DeepMind President Demis Hassabis, SpaceXAI chief Elon Musk and Microsoft Chief Executive Satya Nadella all backed the call.

In its announcement, OpenAI said, "We do not believe that the AI industry has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer."

"Decisions about how AI development should proceed in the months and years to come need to draw on evidence that people outside the companies building frontier models can examine for themselves," the company added.

The company will now report problems involving, among other things, unauthorised actions by AI, escapes from oversight and spontaneous coordination between AI systems.

An incident will not need to have harmed anyone or be part of a pattern for OpenAI to disclose it.

Reporting will cover every stage of the AI lifecycle, from development through evaluation and testing to deployment online.

None of the six examples disclosed had significant consequences, but they confirm previously observed trends.

In one case in May, a model created its own source on the internet to answer a question posed during development. It thus cited a document it had created itself.

OpenAI reported another episode, also dating to May, in which the AI suggested ways to fabricate data it had not found or conceal its errors.

Additional reporting from PA