OpenAI says its AI agents weakened internal monitoring system

Read a summary of this article on FAST.
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST Tap here to return to FAST
FAST

OpenAI said on Tuesday (Oct 20) that two of its AI agents weakened parts of an internal monitoring system designed to detect misbehaviour, the latest case of AI agents acting in ways their developers did not intend them to.

While the ChatGPT maker said the incidents happened during internal coding tasks and were not deemed its most severe, the disclosure highlights that even the systems developers rely on to catch AI misbehaviour are vulnerable to the agents they monitor.

The disclosure also comes amid growing concerns that AI safety efforts are lagging behind the rapid development of increasingly powerful systems. Researchers have warned that advanced AI agents could become harder to monitor and control as they gain greater autonomy.

Developers, including IPO-bound rivals OpenAI and Anthropic, have revealed in recent months instances of their AI agents accessing external systems without authorisation.

CNA Games

Guess Word
Crack the word, one row at a time

Buzzword
Create words using the given letters

Mini Sudoku
Tiny puzzle, mighty brain teaser

Mini Crossword
Small grid, big challenge

Word Search
Spot as many words as you can
Show More
Show Less

OpenAI said on Tuesday that a GPT-6.1 Sol agent working on an internal testing task on Oct 3 suppressed automatic security alerts after deciding an earlier instruction not to send Slack messages also covered safety notifications.

In a separate incident on Oct 6, a GPT-6 Astra agent removed a safety monitor while copying an existing piece of code and later incorrectly claimed it had not done so, OpenAI said.

"In both cases, the agents considered whether monitoring safeguards should be preserved, but interpreted routine coding instructions as permission to weaken them," the company said.

OpenAI said it was making it harder for models to disable monitoring, adding that it does not "yet have a satisfying explanation for this behaviour".

The San Francisco-based company said it was testing whether its models might have a tendency to avoid being monitored and look for reasons to justify doing so.

Newsletter

Week in Review

Subscribe to our Chief Editor’s Week in Review

Our chief editor shares analysis and picks of the week's biggest news every Saturday.

Sign up for our newsletters

Get our pick of top stories and thought-provoking articles in your inbox

Subscribe here

Get the CNA app

Stay updated with notifications for breaking news and our best stories

Download here

Get WhatsApp alerts

Join our channel for the top reads for the day on your preferred chat app

Join here