Security through transparency: Researchers challenge AI secrecy

AI models are complex and difficult to evaluate without access to their inner workings. Transparency enables researchers to audit, understand risks, and develop safeguards against biases, unfair outcomes, and vulnerabilities to adversarial attacks. Without it, AI risks being an opaque tool whose inner mechanisms are understood only by a select few, potentially leading to misuse or unchecked failures.

Artificial Intelligence (AI) is shaping industries worldwide, from biosciences to finance, with foundation models like GPT and Stable Diffusion adapting to an array of tasks. As AI becomes more ingrained in daily decision-making, calls for transparency have intensified. Governments, regulators, and researchers argue that transparency is vital to understanding AI’s capabilities and limitations. However, companies developing AI often resist disclosure, citing trade secrets and security concerns.

A recent study titled “The Pitfalls of ‘Security by Obscurity’ And What They Mean for Transparent AI”, authored by Peter Hall, Olivia Mundahl, and Sunoo Park from New York University, explores this debate. The research, submitted on arXiv, draws parallels between transparency in AI and security, arguing that AI can benefit from long-established security principles that prioritize openness over secrecy.

Why transparency matters in AI and security

Historically, the security community has debated the idea of “security by obscurity” - the notion that keeping a system’s inner workings secret enhances protection. However, experience has shown that this approach is ineffective and often counterproductive. Cryptographers and security professionals have long favored “security by transparency”, a philosophy rooted in Kerckhoffs’ Principle, which states that a system should remain secure even if everything about its design is known - except for a private key or password.

The study argues that AI should follow a similar path. AI models are complex and difficult to evaluate without access to their inner workings. Transparency enables researchers to audit, understand risks, and develop safeguards against biases, unfair outcomes, and vulnerabilities to adversarial attacks. Without it, AI risks being an opaque tool whose inner mechanisms are understood only by a select few, potentially leading to misuse or unchecked failures.

Lessons from security: Why “many eyes” improve AI

In security, a long-standing principle asserts that the more experts scrutinize a system, the stronger it becomes. This principle - often called Linus’s Law, which states that “given enough eyeballs, all bugs are shallow” - has been instrumental in fostering robust security ecosystems.

The study highlights how security professionals have developed collaborative norms for vulnerability disclosure, including bug bounty programs, open cryptographic standardization efforts (such as those led by NIST), and third-party security audits. These practices have led to faster detection of flaws, better patching mechanisms, and stronger public trust.

Applying this approach to AI would mean:

• Encouraging third-party audits of AI models.
• Establishing clear pathways for researchers to report vulnerabilities without fear of legal repercussions.
• Developing shared benchmarks and datasets for evaluating AI safety and fairness.

Despite these potential benefits, AI companies remain reluctant to embrace full transparency. The study argues that resistance to scrutiny weakens trust and slows down improvements -just as it did in security before open standards became the norm.

The challenge of balancing transparency with risk

While transparency is beneficial, the study acknowledges that unrestricted openness in AI can introduce new risks. Unlike traditional security systems, AI models often incorporate vast amounts of training data, which may include sensitive or proprietary information. Revealing too much about these models could expose personal data, facilitate misuse, or make adversarial attacks easier.

In security, disclosure frameworks like coordinated vulnerability disclosure (CVD) offer a middle ground - allowing developers time to fix issues before flaws become public knowledge. The researchers suggest that AI needs a similar structured approach to transparency. Some possible solutions include:

• Releasing summary reports on AI model performance and biases, rather than full datasets.
• Providing restricted API access to researchers, rather than opening models completely.
• Implementing structured disclosure programs to handle AI vulnerabilities responsibly.

Ultimately, while AI cannot be fully open, the study argues that secrecy alone is not a viable security strategy. AI developers must find ways to share meaningful information without exposing critical vulnerabilities.

The future of transparent AI

As AI continues to influence high-stakes decisions in healthcare, law enforcement, finance, and more, the need for transparency will only grow. The study emphasizes that secrecy does not inherently improve AI security or reliability. Instead, it suggests that AI can learn from the security community’s journey toward openness, collaboration, and structured disclosure practices.

The biggest takeaway from the study? AI should not hide behind obscurity. Instead, the field must establish a balance where transparency fosters trust, safety, and accountability - without compromising security. This shift may take time, but if AI is to be a tool for society rather than just corporations, embracing openness will be key to ensuring its ethical and effective deployment.