Anthropic, OpenAI and Google discuss AI testing body
The three labs have met in a working group since July. On the table is an independent body that would test and audit models before release.
What is being discussed
According to The Information, representatives of Anthropic, OpenAI and Google DeepMind have met in a working group since July, most recently last week. The agenda covers shared protocols for testing frontier models before release, independent evaluations and standardised risk assessments. Dario Amodei has been the main force pushing the talks forward.
Where the idea comes from
The idea builds on a July proposal by Demis Hassabis for a US-based frontier AI standards body modelled on FINRA, the self-regulatory organisation that oversees US brokerages. His design pairs government oversight with industry funding and pre-release model assessment.
Open questions
At a company meeting, Sam Altman said he supports a testing and auditing organisation, but that labs would have to build it themselves if the US government does not back it. The companies differ on the state's role: Anthropic favours a partnership with government, while OpenAI puts voluntary standards first. Cost is another question, since compliance burdens could strengthen the largest companies that can most easily carry them.