One AI, Three Different Harms: Why Global Healthcare Rules Are Fracturing

Emotional AI is forcing regulators to confront a problem that conventional healthcare rules were never designed to handle: a system can be clinically safe and privacy-compliant while still fostering dependency, weakening human contact or reshaping care relationships.

In "No universal patient: temporal logics of harm prevention in emotional AI healthcare across Japan, China, and Hong Kong," published in AI & Society, Rosa E. Martín-Peña and Zhengmin Li show that Japan, China and Hong Kong do not simply regulate emotional AI differently. They define the patient, the harm and the moment for intervention differently, producing three distinct governance logics around the same technology.

The same AI can become three different regulatory problems

Much of international medical AI governance has been built around principles such as autonomy, privacy, accountability and data protection. Martín-Peña and Li argue that these frameworks often rest on an implicit image of the patient as an autonomous individual whose rights can be identified and protected through formal rules. East Asian governance traditions, however, do not always construct the patient in the same way.

• Japan is presented as a system in which the patient is understood more relationally, embedded within families, communities and care networks. From that perspective, the danger posed by emotional AI is not confined to data misuse or technical malfunction. Harm can also emerge when machines alter human relationships, encourage dependency or displace forms of care that previously depended on interpersonal contact.
• China takes a different route. The study describes a governance model in which the patient is primarily treated as a subject requiring protection, while emotional dependency can be framed as a population-level risk requiring preventive intervention. China's regulatory architecture therefore places heavier emphasis on obligations imposed before or during deployment, including rules aimed directly at anthropomorphic and emotionally interactive AI.
• Hong Kong, meanwhile, is analysed through a rights-based and juridical model. Privacy, consent, data use and legally enforceable interests sit near the centre of its approach, giving regulators clear tools when personal information is mishandled. Relational harms are harder to capture because unhealthy attachment or erosion of human contact may occur without producing a conventional privacy or legal violation.

An identical emotional AI system can hence appear as a relational technology in Japan, a behavioural intervention requiring preventive controls in China and a data-processing system in Hong Kong. Regulation is not simply applying different rules to the same object. The regulatory system partly determines what kind of object the technology becomes.

The overlooked question is not only how AI is regulated, but when

The study focuses on regulatory timing. Japan, China and Hong Kong are characterised as following iterative, preventive and reactive approaches respectively, creating very different pathways through which risk becomes visible and actionable.

Japan's iterative approach allows norms to develop through professional experience, stakeholder dialogue and gradual adjustment. Such flexibility can be valuable where emerging technologies evolve faster than legislation, but it creates a clear vulnerability: commercial deployment may move ahead of the institutional learning needed to identify emerging harms. Emotional dependency can therefore become visible only after technologies are already deeply embedded in care settings.

China's preventive model attempts to move regulation earlier in the technological lifecycle. The study highlights rules governing anthropomorphic AI that address dependency, data use and simulated family relationships, reflecting an effort to constrain problematic interaction patterns before they become widespread. Preventive regulation can reduce exposure to certain risks, although the authors also identify the possibility that broad restrictions could suppress beneficial applications alongside harmful ones.

Hong Kong's reactive model operates most effectively when harm becomes a recognisable legal event. Privacy violations, improper data collection or breaches of consent can activate formal remedies, but emotional harm may accumulate gradually without ever crossing such a threshold. A patient can become excessively attached to an AI companion without any data law being broken, leaving a potentially serious problem without an obvious institutional owner.

So, timing becomes a form of policy choice. Acting early risks regulating uncertain harms too aggressively; acting gradually can allow commercial pressures to outpace safeguards; acting after violations occur may miss harms that never become legally legible. Emotional AI exposes the weaknesses of all three strategies because its consequences often develop through repeated interaction rather than a single identifiable incident.

Companion robots reveal what conventional AI regulation struggles to see

The study uses PARO, a therapeutic robotic seal used in elderly care, as an analytical case to test these contrasting governance logics. PARO responds to touch and sound and has been associated in earlier research with benefits including reduced anxiety and agitation among dementia patients. Its therapeutic value comes precisely from the emotional connection it is designed to create.

That same feature produces its regulatory difficulty. Emotional attachment may support therapy, but excessive dependency may also displace human interaction, alter family relationships or increase burdens on caregivers. Research discussed in the paper includes examples of strong attachment and also points to wider uncertainty about how relational harms should be measured or reported.

Generative AI makes the problem substantially harder. Newer companion systems can hold open-ended conversations, retain information and adapt their responses over time, creating forms of interaction far more complex than PARO's sensor-based behaviour. Persistent memory and conversational fluency can deepen attachment while making it more difficult to predict how emotional relationships with machines will develop.

Regulators built around clinical safety, consumer protection or personal data therefore face a category problem. Emotional harm may not resemble a faulty medical device, a misleading advertisement or a data breach, even when the consequences for a vulnerable patient are significant. The paper's comparison suggests that sophisticated regulatory systems can remain highly developed while still being blind to entire classes of harm.

Global AI rules may fail if they assume everyone is protecting the same patient

International AI governance increasingly seeks common standards, shared principles and regulatory interoperability. Martín-Peña and Li argue that agreement at the level of abstract principles can hide much deeper disagreement over what those principles actually mean in practice. Beneficence, autonomy or protection can produce different obligations depending on whether the patient is treated primarily as a rights-holder, a protected subject or part of a wider web of relationships.

Developing countries have a particular stake in that debate. Governments adopting foreign AI governance models may also inherit assumptions about family structures, institutional capacity, individual rights and the role of the state that do not necessarily align with local health systems. Regulatory borrowing becomes more complicated when AI is used in eldercare, mental health or other settings where informal caregiving and social relationships strongly influence outcomes.

The authors resist the idea that global governance should simply identify one superior model and export it. Their preferred direction is greater transparency about which harms a regulatory system recognises, what kinds of evidence it considers meaningful and what remedies are available when problems arise. Cross-border governance, in their account, should make those differences visible rather than conceal them beneath universal language.

The paper is a conceptual and comparative analysis rather than an empirical test of which regulatory system produces better health outcomes. Its geographic scope excludes countries such as India, Indonesia and Singapore, while the authors also acknowledge that PARO does not capture the full complexity of emerging LLM-powered companions.

Future studies will need to measure the kinds of harm that existing systems barely record: dependency formation, declining human contact, emotional manipulation and changes in caregiving relationships. Without indicators capable of capturing those outcomes, regulators may continue debating how to control risks they cannot systematically observe. The authors identify such measurement, alongside longitudinal assessment of new regulatory regimes, as a major research priority.