New legal framework aims to contain AI-powered national security threats

While current AI capabilities remain below catastrophic thresholds, the paper highlights the steep acceleration in model risk assessments over recent months. OpenAI’s July 2024 internal review rated GPT-4o as a low-risk model for cyberattacks and CBRN misuse, yet just five months later, its successor model O1 was upgraded to medium risk in both categories. These shifting assessments, Ortega argues, reflect a gap in oversight and preparation that existing laws do not address.

A newly proposed legal framework seeks to establish an AI-specific incident reporting regime aimed at preventing emerging artificial intelligence systems from triggering national security crises. The proposal, authored by legal scholar Alejandro Ortega and released in a recent preprint on arXiv, argues that frontier AI models, those approaching general-purpose capabilities, pose growing risks comparable to nuclear energy, aviation, and pandemic-prone biological research. Ortega calls for the development of a government-led system to detect, contain, and learn from high-risk AI incidents in real time.

The regime would introduce mandatory reporting mechanisms for developers of frontier AI models if those systems contribute to, or enable, events that could compromise public safety or national infrastructure. Drawing on policy precedents from the U.S. Nuclear Regulatory Commission, the FAA, and NIH lab safety protocols, Ortega proposes a three-phase structure for incident management: preparatory safety documentation, rapid-response reporting within 24 hours of an event, and government-enforced remediation to harden future defenses.

While current AI capabilities remain below catastrophic thresholds, the paper highlights the steep acceleration in model risk assessments over recent months. OpenAI’s July 2024 internal review rated GPT-4o as a low-risk model for cyberattacks and CBRN misuse, yet just five months later, its successor model O1 was upgraded to medium risk in both categories. These shifting assessments, Ortega argues, reflect a gap in oversight and preparation that existing laws do not address.

The proposal centers on designating “security-critical sectors” where AI incidents could have outsized impact. These sectors include civilian nuclear power, aviation, dual-use life sciences, and frontier AI itself. Under Ortega’s plan, developers of systems falling into these categories would be required to submit national security cases prior to public deployment, structured documents that argue, with empirical support, that the system does not pose unacceptable risks. If any claim in a security case is later invalidated by an unforeseen event, that incident must be reported.

A hypothetical use case illustrates the urgency: an attacker uses a language model to generate a spear-phishing campaign targeting national power grid operators. Upon discovery, the model’s developer alerts the government within hours. Federal agencies coordinate emergency containment efforts and identify a novel jailbreak technique that enabled the attack. Under the proposed regime, the government would then issue binding security upgrades not only to the responsible developer but across the industry.

This coordinated response framework reflects existing mechanisms used in other high-risk sectors. The FAA requires commercial airlines to report major flight incidents within days. The NRC mandates notification of even minor reactor anomalies that could lead to more serious failures. NIH labs working with dual-use pathogens must report containment breaches within 24 hours. Ortega’s proposal adapts these templates to a domain where the risks are computational, fast-moving, and globally distributed.

Crucially, the paper emphasizes that the reporting requirements would be targeted. Developers of non-frontier models would not be subject to full compliance. Instead, the regime is designed to scale with system risk, reducing burden on low-risk innovation while building capacity for oversight of systems with high destructive potential. The proposal also allows for government access to model internals, documentation, and logs during post-incident investigations, a provision Ortega describes as essential to developing a learning-oriented, rather than punitive, response culture.

While the proposal aligns with current legislative efforts in the U.S., including the Secure A.I. Act and the AI Incident Reporting and Security Enhancement Act, it also addresses critical gaps in existing frameworks. Ortega argues that current laws rely heavily on voluntary disclosure and lack binding procedures for incident containment, threat coordination, and cross-developer security mandates. The report cites the slow response to harmful content generation, disinformation risks, and biosecurity research assistance from open-source models as evidence that self-regulation is insufficient.

Titled "Security-Critical Sectors and AI Risk Containment: A Proposal for a Structured Incident Reporting Regime," the paper concludes by warning that time is short. With leading developers forecasting that general-purpose AI systems may outperform humans across most tasks by 2027, the likelihood of misuse, whether deliberate or accidental, continues to grow. Ortega calls for urgent adoption of structured oversight to mitigate worst-case outcomes before they materialize. He writes that if incident response infrastructure is not built proactively, society may face crises with no ready mechanism to respond.

The proposal’s strength lies in its familiarity: rather than create novel bureaucracies, it draws on decades of regulatory practice from adjacent high-risk industries. Whether lawmakers act in time, Ortega warns, will determine whether the AI revolution is managed or allowed to spiral out of control.