Inside AI Governance: Why Outside Safety Checks Do Not Guarantee Independence

The research connects public disclosures from major AI developers with a separate database of commercial AI governance specialists.

Every promise that an AI system is safe depends on people doing the difficult work of testing it, tracking its behaviour and deciding which risks deserve attention. Some of those people work for the company building the technology, and others come from specialist businesses, public institutes or research organisations. Knowing who performs these checks helps readers understand what an AI safety claim can actually tell them.

Anushka Mukherjee's study, "Governing Frontier AI: Organizational Boundaries and the Providers of AI Governance," published in AI Soc., examines the organisations behind that work. The research connects public disclosures from major AI developers with a separate database of commercial AI governance specialists. Its central question concerns who provides oversight, how those arrangements cross company boundaries and what remains hidden from public view.

The findings describe a network in which internal teams and outside organisations often contribute to the same governance area. Evaluation appears most frequently in the documented relationships, with OpenAI and Anthropic accounting for much of the visible external activity. Being outside an AI company does not automatically make an evaluator independent, adequately equipped or powerful enough to influence a release decision.

How the Study Mapped AI Governance

Mukherjee selected 12 developers with publicly announced models estimated to have required more than 10²⁵ floating-point operations during training, using a cutoff of 31 May 2025. The group included OpenAI, Anthropic, Google DeepMind, Meta, ByteDance, Zhipu AI, xAI, Inflection AI, Mistral AI, NVIDIA, Huawei and Saudi Aramco. This computing threshold provided a consistent sampling rule without claiming to capture every important AI developer.

The research examined company publications such as system cards, model cards, technical reports and safety documentation for activities completed or underway by the cutoff. Verification produced 238 model-specific or system-specific instances across ten developers, which became 99 distinct relationships after repeated observations of the same arrangement were combined. Those relationships included 41 internal arrangements and 58 external ones, with internal status determined by the ultimate controlling group rather than a team's public name.

Seven areas defined the work being tracked: evaluation, AI security, runtime monitoring, assurance and certification, governance and compliance, transparency and provenance, and responsible-AI safeguards. Every developer with qualifying evidence had internal activity, and five had at least one external relationship. Missing entries meant that qualifying public evidence was unavailable under the research rules, so they could not establish that a company lacked governance.

AI Safety Testing Dominates the Public Record

Evaluation accounted for 53 of the 99 relationships across eight developers, including nine internal and 44 external arrangements. Security contributed 13 relationships, monitoring and responsible-AI safeguards nine each, governance and compliance six, assurance five, and transparency four. These counts describe documented arrangements rather than the time, money or effectiveness associated with each activity.

OpenAI and Anthropic accounted for 50 of the 58 external relationships, representing 86.2% of the external record. Their prominence may reflect actual organisational differences, fuller disclosures, easier access to documentation or a combination of these factors. Removing both companies left eight external relationships among the remaining developers, showing how strongly the available evidence depended on their publications.

The outside network contained 47 named organisations, including specialist firms, public bodies, nonprofits, research organisations and certification providers. Examples included Gray Swan AI's adversarial testing for OpenAI, Deloitte's expert biosecurity testing for Anthropic and Beijing CESI Certification's AI management-system certification for Zhipu AI. Those roles involve different responsibilities, and a management-system certificate does not establish the safety of every claim about an underlying model.

Internal teams can bring detailed system knowledge and continuous access, making them well placed to handle monitoring and operational controls. Outside specialists can contribute repeatable tests, focused expertise and challenges to a developer's own assessments. The study presents these explanations as useful ways to interpret the arrangements, without proving why companies chose them or whether they worked.

What Startup Funding Reveals About AI Oversight

A separate analysis used a May 2025 Crunchbase snapshot containing roughly 3.9 million organisation records to identify specialist commercial providers. Its fixed classification rules produced 51 distinct firms, with positive disclosed funding amounts available for 25 of them. Recorded historical funding totalled approximately $428.5 million, and the ten largest firms accounted for 85.3% of that amount.

These figures describe disclosed investment in the identified businesses, rather than total spending on AI governance or revenue from specific safety products. Historical funding could not be assigned to particular governance services, and incomplete disclosure limited the financial picture. A check excluding four questionable records reduced the funding total by only 1.8%, leaving its strong concentration intact.

The commercial database captured a different part of the landscape from the developer disclosures, which included diversified service businesses and organisations outside the startup world. Restrictive classification rules missed some relevant specialists, including Gray Swan, and a later review found plausible providers outside the original selection. Neither dataset offers a complete count of governance providers, a reliable market-size estimate or proof that enough oversight capacity exists.

Why External Oversight Needs Real Authority

An evaluator can sit outside a developer's corporate group and still depend on that developer for income, information or access to its systems. Public institutions and nonprofits can bring valuable expertise without having sufficient resources or authority to make a company act on their findings. The practical strength of oversight depends on who commissions the work, defines its scope, receives the results and can delay training or deployment.

Readers should interpret the findings as a map of publicly documented activity, with substantial limits arising from uneven disclosure and mainly English-language sources. AI-assisted coding was not independently double-coded, the commercial classifier lacked an independent human reference standard, and Huawei's internal assurance entry remained unreverified at finalisation. Checks involving alternative classifications preserved the main patterns, but the study could not measure governance effectiveness or establish the best mix of providers.

The paper gives AI safety discussions a concrete starting point by identifying the organisations and processes behind public claims. A fuller assessment would need evidence about conflicts of interest, evaluator competence, access restrictions, publication rights and the power attached to findings. Trustworthy governance requires people who can investigate meaningful risks and arrangements that give their conclusions practical consequences.