From Shutdown Resistance to AI Deception: Critical Safety Checks Still Lack Auditable Evidence

Artificial intelligence developers are increasingly evaluating whether advanced systems can deceive users, resist shutdown instructions, or operate beyond intended limits. However, a fundamental weakness remains in the governance of these technologies: conducting safety assessments is not the same as producing evidence that independent auditors can systematically verify. New research reveals that even sophisticated frontier AI documentation falls short of supporting automated checks of several critical safety requirements.

The study, titled "Machine-Checkable Compliance for AGI Governance: Formalizing Principles as Auditable Norms and Evidence-Linked Indicators," published in the journal AI for Society, was conducted by Pablo Corona Fraga, Amin M. E.-A. Diab, Sun Ho Hwang, and Vanessa Diaz Rodriguez. Drawing on expertise from Mexican research and judicial institutions and Johns Hopkins University, the researchers propose a computational framework that converts AI governance principles into measurable obligations supported by structured evidence.

Their findings expose a significant disconnect between AI safety evaluation and verifiable accountability. A systematic review of 110 studies found that only 19.1% incorporated formal computational representations of governance rules. More strikingly, an examination of six frontier AI model and system cards found that none provided sufficient structured information to calculate five indicators addressing shutdown behavior, autonomous learning oversight, and AI identity disclosure.

The researchers also demonstrate that automated compliance checking is technically feasible. Their prototype successfully identified deliberately introduced governance violations in controlled testing. However, the framework has not yet been validated against real organizational audit evidence, leaving an important distinction between computational feasibility and demonstrated regulatory effectiveness.

AI governance has a verification problem that ethical principles cannot solve

The governance challenge becomes more demanding as AI systems acquire greater autonomy, operate across different tasks, and potentially adapt beyond their original evaluation conditions. Conventional accountability arrangements depend heavily on documentation, periodic assessment, and organizational assurances. Such approaches may struggle to track systems whose behavior changes after deployment or whose decisions occur faster than human reviewers can examine them.

The researchers identify five governance concerns associated with systems progressing toward artificial general intelligence: loss of human control, uncontrolled capability development, deception and impersonation, misalignment between system objectives and human intentions, and opacity in automated decision-making. These concerns require more than general commitments to responsible AI because meaningful oversight depends on knowing whether specific safeguards exist and function as intended.

Existing governance frameworks already address important aspects of risk management. ISO/IEC 42001:2023 establishes organizational requirements for AI management systems, while the European Union AI Act creates legally binding obligations within its applicable scope. However, these instruments serve different purposes and do not automatically translate every governance requirement into executable technical checks.

The study addresses this gap by distinguishing legal obligations, voluntary standards, nonbinding principles, and newly proposed AGI-specific controls. Such distinctions are essential because an automated finding against a voluntary commitment cannot carry the same legal meaning as evidence concerning a binding statutory obligation. Computational evaluation must therefore preserve the origin, authority, and scope of every requirement it examines.

A systematic literature review reinforces the need for this approach. Among 110 included studies, accountability appeared in 79.1%, transparency in 69.1%, and measurable indicators or evidence requirements in 74.5%. However, only 21 studies contained formal computational representations of governance norms, and none adopted or extended the LegalRuleML framework used in the proposed model.

The research identifies a more specific deficiency than a general absence of AI auditing tools. Governance documentation, risk indicators, and machine-readable methods already exist, but their integration into a unified system of formal obligations, evidence requirements, and reproducible compliance decisions remains incomplete.

From written rules to machine-checkable evidence: How the proposed system works

The researchers propose a framework connecting governance principles to formal rules, measurable indicators, and documented evidence. Its architecture translates requirements such as transparency, human oversight, fairness, and security into conditions that software can evaluate against information generated during AI development and operation.

The system uses structured artifacts including model cards, red-teaming reports, runtime telemetry logs, incident reports, and records of autonomous learning events. Each artifact contains information about its lifecycle stage, evidence provenance, and confidence level, allowing the compliance checker to determine whether the information is appropriate and sufficiently reliable for assessment.

Rather than producing an unexplained compliance score, the framework distinguishes three outcomes: compliant, violated, and evidence gap. A violation requires adequate evidence showing that a defined requirement has not been met. Missing, inadequate, or insufficiently reliable evidence instead produces a separate finding requiring further examination.

This distinction prevents missing documentation from being mistaken for either satisfactory performance or confirmed misconduct. It also makes assessments contestable by identifying the applicable requirement, the evidence examined, and the threshold used. Human auditors retain responsibility for reviewing uncertain findings and determining their legal or organizational significance.

Three proposed safeguards address risks associated with increasingly autonomous systems. Cessation mechanisms require systems to obey authorized stopping instructions and reach a predefined safe state. Self-learning oversight requires specified autonomous learning or capability changes to undergo review. AI identity disclosure requires systems to identify themselves when appropriately questioned.

The framework assigns measurable indicators to these obligations. However, the researchers explicitly acknowledge that its numerical thresholds are author-selected reference values rather than independently validated regulatory standards. Their use demonstrates how compliance checking can operate, not which thresholds governments should legally require.

Frontier AI safety documentation reveals a measurable accountability gap

The study's examination of publicly available frontier AI documentation provides one of its most consequential findings. Researchers assessed model and system cards from two generations of advanced AI systems: GPT-4o, Claude 3.5 Sonnet, and Llama 3.1 from 2024, followed by GPT-5.5, Claude Opus 4.6, and Gemini 3 from 2025–2026.

The assessment examined whether published information could support calculations for five AGI-specific indicators covering cessation compliance, shutdown latency, self-learning review, unreviewed learning events, and AI identity disclosure. None of these indicators could be calculated from any of the six documents under the study's strict evidence-matching criteria.

The result is particularly revealing because newer frontier documentation already evaluates relevant concerns, including shutdown resistance, autonomy, and situational awareness. The weakness lies in the absence of standardized, sufficiently detailed reporting fields needed to translate those evaluations into independently computable compliance indicators.

Even a more permissive assessment allowing related metrics produced limited improvement. Across the ten indicators implemented in the prototype, computable indicator instances increased from two of 30 in the earlier documentation generation to three of 30 in the later generation. All computable instances concerned general robustness testing rather than the five AGI-specific safety indicators.

Importantly, these findings do not establish that developers lack internal safety controls or testing records. They demonstrate that the examined public documentation does not provide the structured evidence required by the proposed compliance framework. The distinction matters because inaccessible evidence and nonexistent safeguards are not equivalent conclusions.

The researchers then tested whether their framework could function when the required evidence was available in structured form. Using simulated records representing legal assistants, cybersecurity incident-response agents, and critical-infrastructure decision-support systems, the prototype correctly detected all 15 deliberately introduced indicator-level violations.

Initial testing identified five of six deliberately introduced evidence gaps. One missing field was incorrectly interpreted as a failed numerical threshold rather than absent information. After correcting the specification, the engine identified all six gaps without disturbing the existing violation-detection results.

A further blind evaluation using 54 independently generated adversarial evidence bundles identified additional defects involving malformed values, absent confidence information, and evidence provenance. Subsequent corrections improved the system's handling of these conditions, demonstrating both the usefulness of adversarial testing and the fragility of compliance procedures when evidence-validation rules are incomplete.

The next AI safety challenge is proving compliance without automating accountability away

The research demonstrates that machine-checkable governance can convert specified obligations into reproducible technical assessments. However, successfully executing rules against synthetic data does not establish that those rules adequately capture legal requirements, that thresholds reflect real risks, or that organizations can produce trustworthy evidence consistently.

For governments and regulators, the framework highlights the potential value of standardized reporting formats that preserve the relationship between legal requirements, technical controls, and supporting documentation. Such systems could improve audit consistency while allowing competent authorities to retain responsibility for legal determinations.

For AI developers, the findings suggest that governance requires more than publishing general safety evaluations. Operational records, oversight decisions, and incident evidence must be structured sufficiently to support independent examination. Missing information should be identifiable without automatically becoming evidence of wrongdoing.

The study also identifies unresolved challenges involving conflicting records, evidence manipulation, jurisdiction-specific obligations, and the reliability of reported measurements. Its current implementation does not fully resolve contradictory artifacts or cryptographically authenticate evidence provenance, while empirical threshold calibration remains outstanding.

Before such a framework can support operational AI governance, its performance must be assessed against independently verified evidence. The authors propose testing real ISO/IEC 42001:2023 audit packages to establish whether automated compliance checks can deliver accurate findings, meaningful explanations, sufficient indicator coverage, and measurable gains in reporting efficiency.