AI safety stuck in technical jargon? A new framework for smarter regulation

One of the core arguments of the policy brief is that AI safety is being narrowly defined as a problem of model reliability and post hoc mitigation. In the International AI Safety Report, the primary focus is on technical risks such as malfunctions, adversarial attacks, bias in model outputs, and risks from unintended AI behavior. While these issues are critical, the policy brief highlights that true AI safety cannot be achieved by only addressing technical vulnerabilities.

AI safety is now a major focus in global policy discussions, with governments and researchers working to manage the risks that come with advanced AI systems. Concerns range from AI-generated misinformation and built-in biases to cybersecurity threats and the effects of automation on jobs and economies. However, much of the conversation around AI safety tends to be highly technical, often emphasizing ways to fix issues after they arise rather than addressing them at the system level. This approach overlooks the bigger picture - how AI interacts with social, economic, and ethical factors.

A recent policy brief, “AI Safety is Stuck in Technical Terms – A System Safety Response to the International AI Safety Report”, authored by Dr. Roel Dobbe of Delft University of Technology, critiques the dominant AI safety discourse and presents an alternative system safety approach. Published in response to the International AI Safety Report (2025) - a collaborative effort involving 96 global experts - the study offers a more integrated and proactive approach to AI governance, one that moves beyond merely fixing AI failures after they occur.

The limits of a technical approach to AI safety

One of the core arguments of the policy brief is that AI safety is being narrowly defined as a problem of model reliability and post hoc mitigation. In the International AI Safety Report, the primary focus is on technical risks such as malfunctions, adversarial attacks, bias in model outputs, and risks from unintended AI behavior. While these issues are critical, the policy brief highlights that true AI safety cannot be achieved by only addressing technical vulnerabilities.

Dr. Dobbe argues that AI systems are inherently sociotechnical - meaning they exist within complex interactions of technology, human decision-making, legal frameworks, and social structures. As such, a purely technical focus ignores broader risks such as:

• Systemic labor market disruptions caused by automation and AI-driven decision-making.
• Geopolitical implications of AI dominance, including economic inequality and control over AI capabilities.
• Environmental concerns linked to the energy-intensive nature of AI training and deployment.
• Erosion of democratic processes through AI-enabled misinformation and algorithmic bias in content moderation.

By limiting AI safety discussions to model reliability and security vulnerabilities, policymakers risk failing to address deeper structural risks that require interventions beyond the technical domain.

System safety: A holistic approach to AI governance

The study introduces system safety as an alternative framework for addressing AI risks. Originally developed in the aviation, energy, and healthcare industries, system safety is a discipline that integrates engineering, organizational design, and regulatory oversight to manage complex technological systems. Instead of fixing failures after they occur, system safety emphasizes designing AI systems in a way that minimizes risks from the outset.

According to system safety principles, AI risks should not be viewed only as failures of individual models but rather as failures of the entire system in which AI operates. This means considering:

• How AI interacts with human users, organizations, and institutions.
• How AI decision-making processes align with regulatory, ethical, and societal norms.
• What systemic vulnerabilities emerge when AI systems are deployed at scale.

For example, rather than merely trying to reduce bias in AI models, system safety would require rethinking how AI-powered decision-making systems are designed, trained, and regulated to ensure fairness at a broader level. Instead of solely mitigating AI-driven misinformation through content moderation, system safety would focus on how AI platforms structure information flows, incentivize engagement, and shape public discourse.

The challenge of general-purpose AI and the “Curse of Flexibility”

One of the key critiques in the policy brief is that general-purpose AI systems pose unique challenges that traditional safety approaches struggle to address. Unlike domain-specific AI models—such as those used in healthcare diagnostics or autonomous vehicles - general-purpose AI models like ChatGPT, Gemini, and DeepSeek operate across multiple industries and use cases. This flexibility, while powerful, creates a fundamental safety problem.

Dr. Dobbe describes this as the “curse of flexibility”, where AI systems become increasingly complex and difficult to regulate because they are designed to perform an indefinite number of tasks across different contexts. In contrast, traditional safety approaches rely on clear operational boundaries - for instance, aviation safety standards are built around known risks in air travel. General-purpose AI, however, lacks such well-defined boundaries, making risk assessment, oversight, and accountability far more challenging.

Another major issue with general-purpose AI is the belief that simply scaling up AI models leads to better performance and safer systems. Many leading AI companies argue that larger models, trained on bigger datasets, will be more reliable and controllable. However, the study warns against blind faith in scaling, noting that:

• Larger AI models require exponentially more energy and computational resources, contributing to climate impact.
• Increased model complexity reduces transparency, making it harder to understand AI behavior.
• Bigger AI systems create single points of failure, increasing systemic risk if widely deployed.

Instead of relying on ever-larger models, the study advocates for smaller, domain-specific AI systems that are easier to govern, audit, and control within well-defined contexts.

A call for public interest AI and stronger governance

The policy brief concludes with a call for a new paradigm in AI governance - one that prioritizes public interest AI over profit-driven innovation. In contrast to Big Tech-led AI safety efforts, which often focus on protecting corporate interests, a public interest AI approach would emphasize:

• Democratic accountability in AI decision-making, ensuring transparency and public oversight.
• Regulatory frameworks that prioritize systemic safety rather than technical fixes.
• Investment in AI safety research that considers broader social and economic risks.
• Development of AI models that align with sustainability, labor rights, and equitable access to AI technologies.

One promising policy mechanism for achieving this vision is the European AI Act and the upcoming General-Purpose AI Code of Practice. The study argues that these regulatory frameworks must move beyond model-level safety tests and instead integrate system-wide safety measures that account for AI’s interactions with economic, political, and social structures.

The path forward lies in breaking free from the narrow technical framing of AI safety and embracing a more integrated, interdisciplinary, and systemic approach to managing AI’s risks and opportunities.