AI reliability debate shifts as models commit irregularities, not hallucinations

Labels such as confabulation or delusion carry clinical implications and anthropomorphize AI by attributing human mental phenomena to statistical models. The study notes that these terms risk stigmatizing people who experience such conditions and misrepresent the nature of AI systems, which cannot experience perception or misperception. Other terms, like fabrication or falsification, imply intent or deception, even though AI systems do not have agency or purpose.

A new philosophical analysis warns that the global debate over artificial intelligence reliability is being distorted by the widespread misuse of the term “AI hallucination,” arguing that the label masks critical differences between simple system errors and deeper breakdowns in meaning that undermine user trust. 

The findings come from the peer-reviewed article “AI lost the prompt!’ Replacing ‘AI hallucination’ to distinguish between mere errors and Trregularities,” published in AI & Society that argues that discussions about AI trustworthiness require clearer conceptual categories rooted in Ludwig Wittgenstein’s distinction between errors that occur within ordinary language and irregularities that violate fundamental certainties.

“AI hallucination” masks critical distinctions

The paper sheds light on the rapid rise of the term “AI hallucination,” which has become common in journalism, research, and public discourse. Despite this widespread adoption, the authors note that the term has been defined inconsistently more than three hundred times across academic literature, suggesting that its meaning is unstable and often misunderstood. They argue that many definitions treat all flawed AI outputs in the same category, even though some failures resemble typical human mistakes while others fall so far outside the boundaries of reason that they signal a deeper conceptual breakdown.

The authors frame this problem through Wittgenstein’s differentiation between errors and irregularities. Errors include misunderstandings, incorrect classifications, or factual inaccuracies that still make sense within human language. These errors can be explained, corrected, and understood because they occur within the shared rules of ordinary discourse.

Irregularities, on the other hand, involve outputs so detached from meaning that they violate basic certainties that humans do not question. These certainties function as the background assumptions that make understanding possible. When AI systems produce outputs that deny these certainties, they create what Wittgenstein described as a grammatical gap. In practical terms, the system stops being relatable and becomes impossible to communicate with in a meaningful way.

The study illustrates this difference through examples documented in previous AI research: misclassified images, fabricated facts, and implausible descriptions represent errors, but responses that deny the existence of a real country or assign biological features to inanimate objects fall into the irregularity category. According to the analysis, these violations destabilize trust in ways that simple errors do not, because they reveal a collapse in the model’s ability to operate within the boundaries of shared human reasoning.

Researchers examine and reject existing alternatives to “hallucination”

The study conducts a systematic evaluation of proposed alternatives to the term “AI hallucination,” arguing that most fail to capture the philosophical difference between errors and irregularities. The authors assess terms like confabulation, delusion, misinformation, stochastic parroting, fabrication, and falsification, explaining that these labels often introduce new problems by implying intention, consciousness, psychological states, or moral failure.

Labels such as confabulation or delusion carry clinical implications and anthropomorphize AI by attributing human mental phenomena to statistical models. The study notes that these terms risk stigmatizing people who experience such conditions and misrepresent the nature of AI systems, which cannot experience perception or misperception. Other terms, like fabrication or falsification, imply intent or deception, even though AI systems do not have agency or purpose.

Technical alternatives also fall short. Phrases like stochastic parroting describe model behavior but do not clarify the difference between normal inaccuracies and deeper semantic breakdowns. Terms like mirage or silver lining rely on metaphors that require lengthy explanations and may confuse users. Even factual error, although clear and accurate, captures only one side of the distinction and does not address irregularities.

The authors develop a set of criteria for evaluating alternatives. A suitable term must avoid implying intentionality or consciousness, must indicate a mismatch with reality, must distinguish between facts and inferences, must not suggest that AI systems are designed for accuracy, and must be easy for users to understand without technical explanation. It must also avoid negative moral judgment and remain memorable enough for broad public adoption.

Using these criteria, the study finds that no existing alternative adequately captures the two-tiered distinction at the heart of Wittgenstein’s framework. The researchers argue that the lack of precision in terminology has perpetuated confusion, encouraged anthropomorphic interpretations of AI behavior, and obscured important questions about trust, reliability, and system boundaries.

New terminology: Errors and irregularities, plus “AI lost the prompt”

To address this conceptual gap, the authors propose replacing the umbrella term “AI hallucination” with two separate labels: error and irregularity. According to their analysis, error is the most appropriate term for ordinary inaccuracies because it is intuitive, widely understood, and applicable to machines without implying human cognitive states. Errors can be factual, logical, or procedural, and they fall into patterns that users can easily recognize and correct.

Irregularity, the second category, captures outputs that violate certainties or background assumptions. These outputs produce nonsense because they break from the language practices that make communication intelligible. The authors emphasize that irregularities are far more damaging to user trust than errors because they signal a breakdown in the shared framework that allows humans to understand AI responses. An AI system that commits irregularities appears disconnected from the structure of meaning itself, making users far less likely to trust or rely on its outputs.

As an additional communicative tool, the researchers introduce the phrase “AI lost the prompt” to describe moments when an AI system generates outputs that no longer follow the logical or linguistic framework implied by the user’s prompt. While the term echoes a colloquial expression, the authors argue that it effectively conveys the nature of irregularities without anthropomorphism. It signals that the system deviated from the expected pathway of reasoning, producing outputs that cannot be reconciled with the conditions of the prompt or the underlying certainties of language.

The study notes that “AI lost the prompt” may not function well as a formal noun but can serve as a practical expression for everyday users to describe irregularities. For formal contexts, the term irregularity remains preferable because it clearly denotes a deviation from expected linguistic norms without implying agency or intent.

These proposals, according to the authors, are not meant to be definitive but rather to advance the debate by grounding terminology in philosophical rigor. They argue that distinguishing between errors and irregularities is essential for developing reliable AI systems and helping users calibrate trust appropriately.