The illusion of AI help: Fast replies, zero progress

Is artificial intelligence (AI) making customer service worse instead of better? A new study suggests that while AI has dramatically improved response times, it may be undermining the very purpose of service itself.

The study, titled "Designing for Trust, Progress, and Dignity: A Conceptual Framework for Reliability, Responsiveness, and Relational Quality in AI-Enabled Service Systems," published in Information, introduces a new framework for rethinking how AI should be designed in customer-facing environments.

The research outlines a set of structural failures unique to AI systems and proposes a 15-principle design model aimed at aligning automation with long-term relationship quality rather than short-term efficiency gains.

AI systems introduce new failure modes that traditional service models cannot address

The research identifies a fundamental mismatch between traditional service design frameworks and the realities of AI-mediated interactions. Earlier models such as SERVQUAL were built around human-delivered services, where errors tend to be visible, bounded, and correctable. On the other hand, AI systems introduce a new category of failure where outputs can be fluent, confident, and entirely wrong at the same time.

This phenomenon, described in the study as plausible error, represents a structural shift in how reliability failures occur. Unlike traditional service mistakes, which customers can easily identify, AI-generated errors often appear credible and authoritative, making them harder to detect and more damaging in high-stakes contexts such as healthcare, finance, and insurance.

The study further identifies a second failure mode labeled the illusion of responsiveness. While AI systems respond instantly, they frequently fail to move the customer closer to resolution. Fast replies that repeat generic answers, ask users to rephrase questions, or cycle through irrelevant responses create a perception of activity without progress. This disconnect between speed and meaningful responsiveness challenges the assumption that faster service automatically improves customer experience.

A third critical issue is relational overclaim, where AI systems simulate emotional understanding or empathy that they cannot genuinely deliver. When chatbots claim to understand customer frustration but fail to resolve the issue or maintain context, the mismatch between tone and capability triggers distrust. The study emphasizes that this is not simply a design flaw but a structural limitation of current AI systems.

Together, these three failure modes redefine how service quality must be understood in the AI era. Reliability is no longer about reducing visible errors, responsiveness is no longer about speed, and relational quality is no longer about adding warmth. Instead, the study argues for a complete reconceptualization of these dimensions to reflect how AI systems actually operate.

RRR Framework introduces 15 design principles for AI-driven service systems

To address these challenges, the study introduces the RRR Design Framework, built around three redefined dimensions: reliability, responsiveness, and relational quality. Each dimension includes a set of five design principles, forming a comprehensive 15-point system for AI service design.

The framework organizes these principles into preventive and recovery strategies, emphasizing that AI failures cannot always be avoided and must be actively managed when they occur.

In the reliability dimension, the study shifts focus from error prevention to transparency. AI systems should communicate uncertainty explicitly, distinguishing between verified and inferred information. This includes labeling outputs based on confidence levels, avoiding overconfident claims, and clearly acknowledging when the system lacks sufficient information. The framework also recommends integrating verification mechanisms for high-risk outputs and designing systems that route uncertain cases to human experts.

The responsiveness dimension redefines performance metrics away from response speed toward resolution progress. Systems should preserve context across interactions, ensuring that customers do not have to repeat information when switching channels or escalating to human agents. The framework also emphasizes the importance of visible progress signaling, where AI systems clearly communicate what step is being taken and what will happen next. Detecting when users are stuck in repetitive loops and triggering escalation is identified as a critical feature of effective AI systems.

In the relational quality dimension, the study challenges the widespread assumption that AI systems should mimic human behavior. Instead, it argues for transparency about AI identity, calibrated use of human-like features, and preservation of user agency. Systems should allow customers to understand and override decisions, rather than guiding them through opaque processes. The framework introduces the concept of strategic non-humanness, where the non-human nature of AI is positioned as a benefit, particularly in sensitive contexts such as health disclosures or financial distress.

The study's governing principle, described as automating to protect relationships, reverses the conventional logic of AI deployment. Rather than using AI primarily to reduce costs and increase efficiency, the framework positions customer trust, progress, and dignity as the primary design objectives.

Human–AI collaboration requires structural rethinking, not incremental fixes

The research makes clear that improving AI service systems is not a matter of incremental adjustments but requires a structural redesign of how automation is integrated into customer journeys. The interaction between the three dimensions is critical. Systems that perform well in one dimension but fail in others can still produce poor outcomes.

For example, a system that delivers accurate information but fails to move the customer toward resolution may be technically reliable but functionally ineffective. Similarly, a system that resolves issues efficiently but limits user control can undermine trust by reducing perceived autonomy. The study warns that systems combining warmth with unreliable outputs may be the most damaging, as they create a perception of care that is not supported by actual performance.

The framework also highlights the importance of sequencing in design investment. Relational features such as empathy and personalization are unlikely to improve user experience if the system lacks reliability and responsiveness. In such cases, these features may be perceived as superficial or manipulative rather than helpful.

The study provides practical scenarios to illustrate how the framework operates in real-world settings. In a telecommunications billing dispute, the AI system combines data retrieval with uncertainty communication, clearly explains its actions, and resolves the issue while preserving context for escalation. In a healthcare screening scenario, the system uses its non-human identity to reduce user anxiety, avoids making diagnostic claims, and routes ambiguous cases for human review. These examples demonstrate how the three dimensions interact to produce effective outcomes.

The effects extend to hybrid human–AI systems, where coordination between automated and human agents becomes a central design challenge. Effective systems must ensure seamless handoffs, shared context, and clear boundaries between what AI can and cannot handle. The study argues that human agents should be integrated as part of the system architecture rather than treated as a fallback option.

Shift in metrics and governance signals new direction for AI service systems

The study calls for a fundamental shift in how organizations measure AI performance. Traditional metrics such as response time, deflection rates, and cost savings are insufficient to capture customer experience in AI-mediated systems. Instead, organizations should prioritize resolution rates, customer re-contact frequency, and measures of trust and perceived progress.

Relational failures often go undetected in standard feedback systems. Customers who feel dismissed or manipulated are more likely to disengage silently rather than file complaints. This creates a hidden layer of customer churn that conventional metrics fail to capture, making it difficult for organizations to identify and address underlying issues.

The framework also highlights the need for proactive design strategies. Rather than deploying AI systems and responding to failures after they occur, organizations should design against known failure modes from the outset. This includes building mechanisms for uncertainty communication, context preservation, and escalation into the system architecture.

The study further identifies several areas for future research, including the development of new measurement scales for AI service quality, the role of customer familiarity with AI in shaping expectations, and the conditions under which non-human system design enhances or undermines user experience.