OpenAI on Monday announced it has signed agreements with four independent evaluators to help support the artificial intelligence company's safety work.
The evaluators, Apollo Research, METR, Transluce and Goodfire, will initially conduct distinct pilot assessments around model behavior and safeguards, OpenAI said. The company will give the organizations "deep access" to its internal systems as part of the agreements, adding that it plans to use what it learns to expand its work with independent experts going forward.
"We expect this first phase of assessments to lead to longer-term, consistent embedded engagements over time that can continuously provide third party safety signals and recommended improvements on our most load bearing safety claims and safety cases," OpenAI said in a blog post.
Independent evaluators have occupied a quiet corner of the multi-trillion-dollar AI industry thus far, but they've taken on outsized importance following a series of high-profile safety and security incidents this year. The nascent ecosystem is still ironing out best practices, including how its work should be funded and what the reporting structures should ultimately look like.
OpenAI CEO Sam Altman pledged to grant independent evaluators "employee-like access" to the company in September, shortly after Anthropic CEO Dario Amodei said his company would do so as part of his call to slow down the pace of advanced model development. Anthropic has since announced agreements with evaluators including Accenture and METR, and it said in October that it's engaging in discussions with others.