Anthropic cofounder wants brake pedal for AI, says youth can keep up with pace by getting a hobby

Anthropic co-founder Jack Clark has renewed calls for a way to pause AI development if needed. He said governments should prepare rules now as future systems could help build their own successors.

In Short

• Anthropic wants the ability to pause AI development if needed
• Jack Clark says the AI industry currently lacks a brake
• Anthropic has warned future AI could help build new AI

Just after Anthropic demanded the ability to pause AI development, warning that future AI systems could become capable of designing, building and training their own successors with little or no human involvement, the company's co-founder has reiterated the call for stronger safeguards.

Speaking to BBC Newsnight, Anthropic co-founder Jack Clark said the AI industry needs a way to slow down development if necessary, arguing that current AI progress is accelerating without sufficient checks in place.

"You want the option to be able to take your foot off the gas and put your foot on the brake," Clark said. "Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal."

The warning is notable because it comes from one of the companies building some of the world's most advanced AI models. Anthropic is the creator of Claude, a chatbot that competes with OpenAI's ChatGPT and Google's Gemini.

But why is the company asking for caution?

Why Anthropic is concerned

The concern centres on a future scenario in which AI systems become capable of helping design, build and train their own successors. In simple terms, today's AI could eventually help create tomorrow's AI, reducing the amount of human involvement needed in the process.

Anthropic recently argued that its Claude AI is already increasingly assisting with AI research and development tasks. The company warned that future systems could become capable of designing and training new AI models with little or no human input.

To be clear, Anthropic says that moment has not arrived yet. The company also acknowledges that it may never fully happen. However, it believes governments and regulators should start preparing now rather than waiting until the technology reaches that stage.

"We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for," Anthropic said in a recent blog post.

The fear of AI creating better AI

For ordinary users, the idea may sound like science fiction. Imagine a software engineer creating a smarter engineer, who then creates an even smarter one. If AI systems begin improving future AI systems, technological progress could accelerate dramatically.

That possibility is what worries Clark. He argues that humans must remain in control through government oversight and clear regulations.
"The world needs to do some thinking and we need to eventually develop some new regulations that allow us to be confident in these systems," he said.

Clark compared the current AI boom to the rise of the oil industry more than a century ago. Back then, governments eventually created rules and regulatory frameworks that allowed society to benefit from oil while managing its risks. He believes AI will require a similar approach.

AI could disrupt jobs, but creativity may remain an advantage

Clark also pointed to another potential risk: disruption to the economy. According to him, AI technologies such as "agents", individual AI bots that can perform routine tasks with a degree of autonomy, could eventually take over certain jobs.

The concern comes at a time when major technology companies have conducted mass layoffs over the past year, often pointing to the growing capabilities of AI tools that can perform work previously handled by hundreds or even thousands of software engineers. Despite those concerns, Clark believes humans may still hold an important advantage. He said people who are creative and capable of generating better ideas could be better positioned in an AI-driven economy.

"There are open questions about whether AI systems can be truly creative there is not really evidence for that yet," Clark said. "At Anthropic, we're now limited more by the ability to generate good ideas than the ability to do the engineering to turn those ideas into reality."

For young people worried that an economy increasingly shaped by AI may not have a place for them, Clark offered some unconventional advice: develop hobbies and pursue a liberal arts education.

"People that are creative and can think broadly, people that read a lot, people that have interests are the ones most benefited by this," Clark said. "Indulge in curiosity and it pays back in how you can use this technology."

Clark's comments about hobbies also sparked discussion on X. Responding to a user who said it was surprising to hear such advice from someone who previously worked at The Register, a publication known for its sceptical coverage of technology claims, Clark clarified that his remarks were being interpreted incorrectly.

According to Clark, his advice is not rooted in fears about AI replacing people, but in a belief that those working in technology should have interests beyond computers. He said he routinely encourages new employees at Anthropic to develop hobbies that do not involve technology and, as he put it, to "touch grass" occasionally rather than spending all their time focused on tech.

- Ends