Anthropic grants outside evaluator METR employee-level access in AI safety push
· CNA · JoinRead a summary of this article on FAST.
Get bite-sized news via a new
cards interface. Give it a try.
Click here to return to FAST Tap here to return to FAST
FAST
Oct 15 : Anthropic has given nonprofit METR employee-level access to its systems and is in talks with others including Hugging Face, as AI companies pitch external evaluators as a way to safely advance the technology without slowing its development.
The developer of Claude models is also working with biosecurity nonprofit SecureBio and Accenture's AI business Faculty as evaluators, it said in a blog post on Thursday.
Reuters exclusively reported earlier this week that Anthropic and OpenAI had sought significant control over what outside reviewers could publish, concerned that the findings could leak trade secrets.
A person familiar with the matter told Reuters that talks with Anthropic had evolved recently, with the company becoming more open in negotiations.
CNA Games
Guess Word
Crack the word, one row at a time
Buzzword
Create words using the given letters
Mini Sudoku
Tiny puzzle, mighty brain teaser
Mini Crossword
Small grid, big challenge
Word Search
Spot as many words as you can
Show More
Show Less
A key sticking point between evaluators and AI labs is that information the companies might consider crucial intellectual property can also be relevant to assessing a model's risks, leading to disputes and redactions, a separate person familiar with the discussions told Reuters.
Experts have also said the independence of evaluators is a concern, as Anthropic and OpenAI choose who gets to carry out the reviews.
The outcome of the talks will help determine how the world scrutinizes a technology that some of its creators warn may be advancing beyond humanity's ability to manage.
THE METR TIE-UP
Anthropic said on Thursday six employees of METR — or Model Evaluation and Threat Research — have company hardware, office space and access to internal systems roughly equivalent to its risk assessment teams. However, customer data, legally privileged information and model architecture will remain off limits.
The company said it cannot veto METR's findings on risks, incidents and practices, but can require narrowly defined redactions. METR can also publicly question those redactions without revealing the withheld information.
"In line with our transparency policies, our report will openly describe the scope, access, and other limitations of our work," METR said. The nonprofit had proposed on Wednesday that developers allow private staff interviews, timely publication of assessments and transparency about access limits and redactions.
Anthropic said its other evaluator partnerships are at an early stage, with more limited access and publication rights.
The company is developing three tiers of access. Evaluators at the first level will conduct narrowly scoped investigations and need approval to publish. Those at the second will receive more information and can publish without approval, subject to redactions evaluators can publicly challenge.
The third, used by METR, provides employee-like access with similar publication rights to the second tier.
SecureBio is in the first tier and is reviewing the chemical and biological weapons chapter of Anthropic's August risk report, having published a pilot review of the previous report. Anthropic said it would consider expanding SecureBio's access and publication rights after the review.
Anthropic also announced last month that it is working on a tie-up with Faculty covering model evaluations, adversarial testing and assessments of whether models behave as intended, among other areas.
Newsletter
Week in Review
Subscribe to our Chief Editor’s Week in Review
Our chief editor shares analysis and picks of the week's biggest news every Saturday.
Sign up for our newsletters
Get our pick of top stories and thought-provoking articles in your inbox
Get the CNA app
Stay updated with notifications for breaking news and our best stories
Get WhatsApp alerts
Join our channel for the top reads for the day on your preferred chat app