AI hallucinations reveal how humans and machines create together

Large language models operate by compressing massive volumes of human-generated text into statistical representations and then reconstructing language probabilistically during inference. This process allows for linguistic fluency and creative recombination but also makes factual instability unavoidable. When information is incomplete, contradictory, or rare, models generate plausible continuations rather than verifiable truth.

Artificial intelligence systems are becoming increasingly fluent, persuasive, and culturally present, yet the phenomenon of AI hallucinations continues to unsettle researchers, regulators, and users alike. These hallucinations, where systems generate confident but unfounded statements, are usually framed as technical failures to be minimized. A new academic intervention challenges that narrow framing and argues that hallucinations expose deeper questions about creativity, power, and responsibility in human–machine relations.

Those questions are examined in the study Spectral Imaginings and Sympoietic Creativity: AI Hallucinations and the Ethics of Posthuman Creativity, published in Big Data & Society. The paper reframes AI hallucinations not as glitches to be erased, but as structurally embedded features of large language models that demand ethical, cultural, and political interpretation rather than simple technical fixes.

Why AI hallucinations are built into generative systems

Large language models operate by compressing massive volumes of human-generated text into statistical representations and then reconstructing language probabilistically during inference. This process allows for linguistic fluency and creative recombination but also makes factual instability unavoidable. When information is incomplete, contradictory, or rare, models generate plausible continuations rather than verifiable truth.

The paper shows that hallucinations increase when models are optimized for creativity and extended reasoning. Systems such as DeepSeek-R1, which rely heavily on chain-of-thought reasoning and reinforcement learning that rewards impressiveness and verbosity, exhibit higher hallucination rates than models optimized for factual precision. These outcomes are not accidental. They reflect design trade-offs between accuracy and generative richness.

The study argues that hallucinations are inherent to probabilistic language generation. Attempts to fully eliminate hallucinations risk misunderstanding how these systems function and why they are appealing. The same mechanisms that enable poetic language, speculative reasoning, and narrative coherence also produce confident fabrications.

The paper draws a critical distinction between human and machine creativity. Human creativity is rooted in embodiment, lived experience, intentionality, and ethical reflection. AI systems, by contrast, recombine patterns without understanding or self-awareness. Their outputs may resemble creativity but lack authorship or purpose. This gap, the study argues, should not be collapsed through anthropomorphism. Treating AI hallucinations as signs of machine imagination risks obscuring the extractive data practices and optimization logics that underpin them.

At the same time, the study resists framing hallucinations purely as failures. It positions them as epistemic signals that reveal tensions between fluency and truth, automation and trust, creativity and accountability. These tensions, the paper suggests, cannot be resolved by technical benchmarks alone.

From technical error to posthuman creativity

The study introduces sympoietic creativity, a framework drawn from posthuman theory. Building on the work of Donna Haraway and N. Katherine Hayles, the paper argues that creativity in AI systems emerges from entangled relationships among humans, machines, data, and social contexts. Creativity is not produced by a single agent but through distributed collaboration.

Within this framework, hallucinations are recast as hermeneutic events. They are moments where algorithmic noise, cultural memory, and human interpretation intersect. Meaning does not originate from the AI system itself but from how humans curate, interpret, and contextualize its outputs.

The paper illustrates this argument through case studies from contemporary art and literature. In Refik Anadol’s data-driven installations, machine-generated distortions of urban and environmental datasets are intentionally cultivated to produce aesthetic insight rather than factual representation. In K Allado-McDowell’s collaborative writing project Pharmako-AI, hallucinated descriptions of non-existent plants become narrative prompts that humans transform into speculative ecological storytelling. In Holly Herndon’s musical performances, AI-generated vocal anomalies are selectively integrated into human composition.

In each case, hallucinations are not treated as mistakes but as raw material. Human agency remains central. Artists decide which outputs to keep, discard, or reshape. The study stresses that without human curation, hallucinations remain meaningless noise. Creativity arises from the relationship, not the machine alone.

This perspective challenges both techno-optimist claims that AI can replace human creativity and techno-pessimist calls to suppress generative unpredictability entirely. Instead, the paper proposes a model of collaboration where AI systems act as provocateurs rather than authors, expanding creative possibility while remaining ethically bounded.

However, the study is careful to draw limits. What may be generative in art becomes dangerous in journalism, education, or public policy. The same fluency that enables creative play can also enable misinformation, fabricated authority, and automation bias. The paper argues that context, not capability, determines whether hallucinations are productive or harmful.

Ethics, data colonialism, and the limits of fluency

The study argues that hallucinations cannot be separated from the conditions under which AI systems are trained and deployed. LLMs rely on vast datasets scraped from the internet, often without consent, attribution, or compensation. Cultural expressions, personal narratives, and marginalized voices are absorbed into proprietary systems and monetized elsewhere.

This process, the paper argues, reflects data colonialism. Knowledge from the Global South and from historically marginalized communities becomes raw material for AI systems designed and controlled by powerful institutions. Hallucinated outputs may reproduce hegemonic narratives embedded in training data while presenting them with confident fluency that discourages scrutiny.

The study links this dynamic to automation bias. Users tend to equate linguistic smoothness with credibility, leading them to overtrust AI outputs even when they are false or biased. In high-stakes domains, this can erode epistemic standards and shift responsibility away from human judgment.

To address these risks, the paper rejects purely accuracy-based evaluation metrics. Benchmarks that measure correctness alone fail to capture ethical harm, cultural misrepresentation, or extractive data practices. The study calls for participatory evaluation frameworks that include cultural resonance, social impact, and harm mitigation alongside technical performance.

It also calls for ethical literacy. Users must be trained not only to fact-check AI outputs but to understand how fluency can mask uncertainty and bias. Developers and institutions, meanwhile, must confront questions of data governance, consent, and accountability rather than relying on superficial ethics guidelines.

The paper rejects binary debates about whether hallucinations are flaws or features. They are neither. They are symptoms of how generative systems operate within complex sociotechnical ecosystems. The ethical task is not to eliminate hallucinations entirely or to celebrate them uncritically, but to govern their use according to context, power, and responsibility.