AI mental health support often promises more than it can deliver

AI mental well-being systems are being deployed without clear standards for what they promise, how they work, or what risks they introduce, warns a new study published in the Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems.

Titled Framing Responsible Design of AI Mental Well-Being Support: AI as Primary Care, Nutritional Supplement, or Yoga Instructor?, the research conducts an in-depth analysis of non-clinical AI mental health tools. Rather than evaluating performance or accuracy alone, the study reframes responsibility as a design and policy problem rooted in how AI systems are positioned, marketed, and understood by users.

Redefining responsibility in AI mental well-being tools

The research is based on two complementary methods. First, the authors conducted in-depth interviews with 24 experts spanning human-centered computing, clinical psychology, psychiatry, medical ethics, health policy, and AI governance. Second, they analyzed more than 100 U.S. regulatory and policy documents related to health, wellness, and digital technologies. This combined approach allowed the authors to map how responsibility is currently defined in law and practice and where AI mental well-being tools fall outside existing oversight structures.

A key finding is that AI mental well-being systems are often treated as a single category, despite vast differences in what they claim to provide. Some tools present themselves as general wellness aids, while others implicitly suggest therapeutic benefits without meeting clinical standards. This ambiguity, the study argues, creates ethical and safety risks because responsibility should scale with the benefits a system claims to deliver.

To address this, the authors introduce a typology that reframes AI tools through familiar analogies. Some systems function like nutritional supplements, offering broad well-being support without guarantees of effectiveness. Others resemble over-the-counter medications, claiming relief for specific symptoms but operating without rigorous clinical validation. A third category mirrors yoga instructors, providing guided practices without responsibility for outcomes. The most consequential category aligns with primary care providers, where systems promise evidence-based interventions and therefore require strong safeguards, evaluation, and escalation pathways.

The study shows that responsibility is not an abstract principle but a function of design intent. The more an AI system positions itself as capable of improving mental health outcomes, the higher the ethical and regulatory burden it must meet. Many current tools, the authors find, implicitly claim benefits associated with higher-risk categories while avoiding the corresponding responsibilities.

Active ingredients, guarantees, and hidden risks

The study identifies three core criteria that should govern responsible AI mental well-being design. The first is clarity around guaranteed benefits. Developers must explicitly state what a system can and cannot do, avoiding vague claims that blur the line between wellness and treatment. Without clear guarantees, users may overestimate a tool’s capabilities, particularly during periods of emotional vulnerability.

The second criterion concerns active ingredients. In traditional healthcare, interventions are evaluated based on known mechanisms such as cognitive behavioral therapy or mindfulness-based stress reduction. The study finds that many AI tools fail to specify which psychological principles they rely on or how these are operationalized in system behavior. This lack of transparency makes evaluation difficult and obscures potential failure modes.

The third criterion is proportional risk. The authors argue that risk tolerance should align with promised benefits. Tools offering general well-being guidance may tolerate minor inaccuracies, while systems claiming to address anxiety, depression, or crisis situations must meet far stricter safety standards. Yet the analysis reveals that many AI tools expose users to risks such as emotional distress, delayed help-seeking, over-reliance, and social withdrawal without adequate safeguards.

A key concern highlighted in the study is that AI tools can unintentionally displace human support. Users may substitute AI interactions for professional care or social relationships, particularly when systems present themselves as always available, empathetic, and nonjudgmental. Over time, this displacement effect could exacerbate isolation or delay intervention for serious conditions.

The authors also point to ethical tensions that remain unresolved. One debate centers on whether population-level benefits can justify severe harm to a small number of users. While some experts argue that broad access to low-cost support is valuable, others caution that even rare failures can carry unacceptable consequences in mental health contexts. The study does not resolve this debate but emphasizes that design decisions must confront it explicitly rather than ignoring it.

How this impacts policy, developers, and public trust

Current regulatory frameworks are poorly equipped to handle the hybrid nature of AI mental well-being tools. Many systems avoid classification as medical devices while still influencing mental health behaviors. This regulatory gray zone leaves responsibility largely in the hands of designers and companies, raising concerns about incentives and accountability.

For developers, the research calls for a shift away from feature-driven innovation toward responsibility-driven design. Systems should be built with explicit boundaries, transparent mechanisms, and escalation pathways to human care when risk increases. Design choices such as disclaimers, onboarding language, and conversational framing are not neutral, the authors argue, but central to how users interpret and rely on AI support.

For policymakers, the findings suggest the need for tiered oversight models that reflect different levels of claimed benefit and risk. Rather than applying one-size-fits-all regulation, frameworks should scale requirements based on how closely an AI tool resembles a clinical intervention. This approach could preserve innovation in low-risk wellness applications while imposing necessary safeguards on higher-stakes systems.

The study also has consequences for public trust. As AI becomes embedded in intimate aspects of life, trust will depend less on novelty and more on reliability, transparency, and accountability. Tools that overpromise and underdeliver risk eroding confidence not only in individual products but in AI-assisted care more broadly.

Responsible design does not mean rejecting AI in mental well-being. Instead, it requires acknowledging that psychological support is inherently relational, contextual, and risk-sensitive. AI can play a role, but only when its limitations are clearly communicated and its use is situated within a broader ecosystem of care.