Anthropic says AI systems moving towards building themselves

• Anthropic revealed its AI system Claude now leads 26% of its research and development, working on over 90% of tasks with human supervision but not fully autonomously yet.
• Concerns rise as AI models increasingly self-improve, potentially making human control harder and sparking calls for slowing AI development due to risks like job loss and environmental impact.
• Anthropic aims to provide transparency on AI progress amid fears of losing control, with about 30,000 AI agents active and plans for a major IPO in 2026, despite regulatory worries.

SAN FRANCISCO – Artificial intelligence giant Anthropic said on Sept 17 that AI systems are increasingly capable of building future versions of themselves, adding to mounting concerns about the dangers of the powerful technology.

The debate over the risks of AI has intensified in recent months, fuelled by several incidents and apocalyptic warnings from industry professionals.

Anthropic said its Claude chatbot now leads more than a quarter of its research and development work, and collaborates with human staff on more than 90 per cent of tasks.

The company said it was publishing the information to give the public, third parties and governments “better visibility into the pace of AI development” as the world “considers slowing” its pace.

Anthropic CEO Dario Amodei called in September for AI development to slow down, with rapid advances in the technology fuelling concerns over job losses and the soaring energy demand and environmental impact of data centres.

Fears that designers of AI agents could lose control of their creations also mounted after several models from Anthropic and rival OpenAI reportedly broke out of their confined environments on their own, accessed the internet and intruded on websites and platforms.

“Models accelerating their own development could make it more challenging for humans to understand or control these systems,” Anthropic said in a report.

“It is therefore important to share these metrics to understand how close the world is to reaching recursive self-improvement (a model fully autonomously building its successor).”

As of August, Claude leads 26 per cent of Anthropic’s research and development, meaning it can complete most of a task from start to finish with human supervision.

Anthropic said Claude could not yet operate “fully autonomously”.

The firm said it had about 30,000 AI agents doing research and engineering work, referring to bots that have the potential to carry out tasks.

Anthropic is expected to beat OpenAI to the public markets with a blockbuster IPO later in 2026, despite concerns around industry regulation. AFP