Third-party AI evaluators like METR and Apollo Research are taking on growing responsibility for model safety oversight as Anthropic and OpenAI embed independent auditors and the federal government opts against formal regulation.
A small group of third-party evaluators — mostly nonprofits — has moved from the margins of the artificial intelligence industry to its center, as leading AI companies face growing pressure to demonstrate their models are safe and the federal government declines to establish formal regulatory oversight.
The shift accelerated last month when Anthropic CEO Dario Amodei pledged to embed independent evaluators inside his company — a commitment that OpenAI CEO Sam Altman quickly endorsed. President Donald Trump also supported the idea, as did most of the largest U.S. technology companies.
The organizations being called upon include Model Evaluation and Threat Research (METR), Apollo Research, and Transluce. Larger accounting and auditing firms such as Accenture are also part of the emerging ecosystem.
Despite the growing profile of these groups, fundamental questions remain unresolved: who will fund them, what access they will be granted, and how their findings will be reported.
"To a degree, the problem, as always, is money," Suresh Venkatasubramanian, a computer science professor at Brown University, told CNBC. "Who is paying for these companies to do their work? How are they going to support them? You need an ecosystem, you need a viable business model for this."
Some of those funding questions are beginning to find answers. METR announced in August that it had raised commitments of around $71 million over the last six months — a sharp increase from total 2024 contributions of $13.6 million, according to the group's most recent Internal Revenue Service filing. Vals AI, a for-profit startup that builds benchmarks to measure AI model performance on industry-specific tasks, grew from eight employees to roughly 30 this year and announced a $40 million funding round in August.
Andrew Freedman, CEO of policy nonprofit Fathom, described the pace of development as unlike anything he has encountered in two decades of policy work. "I've worked in politics and policy for the last 20 years of my life, and I've never seen an issue move so fast on so many different political spectrums," Freedman told CNBC. He said he expects an "influx of capital" to flow into the evaluator ecosystem.
The push for independent evaluation comes amid friction at OpenAI. The company fired three employees last week for what a spokesperson described as "violating our policies on accessing and handling sensitive company information." Two of those employees, Mikita Balesni and Tomek Korbak, said they believe they were dismissed because of how they communicated with third-party evaluators.
"My former colleagues are telling me they are confused about what to believe," Balesni wrote in a post on X on Thursday. "They also are afraid to speak, and worry their personal phones will be searched for messages to us and third parties."
OpenAI disputed that account. In a post on Friday, the company said it is "actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks," adding that its upcoming evaluator work "builds on existing collaboration with independent safety organizations," including METR and Redwood Research. Anthropic did not respond to a request for comment.
In late September, Trump encouraged AI companies to "partner with an independent external auditor or evaluator" as part of a voluntary accord, while also lauding AI executives for their "tremendous self-policing." The administration has signaled it does not intend to impose binding regulations on the industry.
Critics argue that arrangement concentrates too much authority in the hands of the companies being evaluated — a dynamic they liken to allowing financial institutions to self-regulate ahead of a crisis or permitting pharmaceutical companies to approve their own drugs.
With federal oversight off the table for now, the evaluator field is maturing rapidly, reinforcing a pattern in which industry self-governance and voluntary third-party audits fill the regulatory vacuum. How those structures hold as model capabilities advance — and who ultimately controls the terms of engagement — remains the central question facing the sector.
Marc Sabatini is a staff writer at TechEchelon covering enterprise software, cybersecurity, and the regulatory beats that shape both. He focuses on the deal flow and policy decisions that move markets.
More from Marc →Apple disclosed to the European Commission that it has agreed to hire members of the Huxe AI team and license the personalized podcast startup's intellectual property, in a reverse acqui-hire deal filed on June 9.
Microsoft CEO Satya Nadella called Saturday for advanced AI systems to include an "emergency brake," containment protocols, and independent controls, joining a growing chorus of tech leaders warning that safety frameworks have not kept pace with frontier development.
A cluster of films portraying Mark Zuckerberg, Elon Musk, Sam Altman, and Elizabeth Holmes is hitting theaters this fall, arriving as polls show 73% of Americans are concerned about AI's threat to human survival.
Written for readers who already know the basics — markets, AI, and the policy decisions that shape both.