News

OpenAI Launches Embedded Assessments with Independent Evaluators

OpenAI partners with Apollo Research, METR, Transluce, and Goodfire for embedded assessments to enhance AI model alignment and safety standards.

OpenAI Launches Embedded Assessments with Independent Evaluators

OpenAI has announced the launch of its first phase of embedded assessments, partnering with four independent evaluators—Apollo Research, METR, Transluce, and Goodfire. This initiative aims to enhance transparency and safety in artificial intelligence by providing external experts with deep access to OpenAI's internal models and systems. The program is designed to evaluate model alignment, safeguard effectiveness, and monitoring systems over extended periods.

Independent Evaluation Framework

Embedded assessments allow external experts to access critical resources such as model checkpoints, source code, training and evaluation data, monitoring systems, and technical documentation. These evaluators also engage directly with OpenAI researchers and engineers to gain insights into the models' development and deployment. This comprehensive access enables assessors to perform independent evaluations and identify potential failures that may not have been anticipated internally.

The initial phase focuses on assessing alignment and safeguards during model training and internal use. These efforts align with OpenAI's commitments under the White House Accord on Super Intelligence, which emphasizes independent safety assessments and public transparency. The initiative also builds on OpenAI's prior experience with third-party evaluations, including a recent pilot assessment of its internal monitoring systems and an investigation into the Hugging Face incident.

Key Contributions from Partner Organizations

Apollo Research: Apollo Research will investigate whether OpenAI's models could covertly work against their developers' intentions. Their assessment will cover four key claims: that training did not incentivize subversion, that strong evaluations do not elicit subversion, that models have not attempted subversion during internal use, and that reasoning behind subversion would be detectable. Apollo will use evidence from training checkpoints, safety testing, and flagged behaviors to conduct its analysis.

Goodfire: Goodfire will focus on evaluating the robustness of OpenAI's safeguards and monitoring systems. Specific details about their methodology and focus areas were not disclosed in the announcement.

The contributions of METR and Transluce were also highlighted, though specific roles and methodologies were not detailed in the initial announcement.

Strategic Implications

By collaborating with independent experts, OpenAI aims to establish stronger shared standards for AI safety and foster greater public trust in the industry. The insights gained from these assessments are expected to inform future safety protocols and contribute to the broader development of transparent and accountable AI systems.

Sign in to save this article Free account: build your research collection and return to read it anytime.