When AI Takes Control but Doctors Take Responsibility: Healthcare's Emerging Automation Paradox

Artificial intelligence is approaching a consequential transition in healthcare, moving from systems that assist clinicians with decisions toward technologies capable of independently coordinating and executing complex workflows. While this evolution promises to reduce administrative burdens and improve care delivery, it also introduces a troubling possibility: healthcare professionals could become responsible for actions increasingly performed beyond their direct control.

A study titled "Healthcare Professionals' Perspectives on Agentic Artificial Intelligence in Clinical Work: A Qualitative Study," published in the journal Healthcare, investigates how medical professionals anticipate this transformation. Authored by Stefanie Mache, Lena Klimaschewski, Monika Bernburg, David A. Groneberg and Volker Harth, researchers affiliated with University Medical Center Hamburg-Eppendorf and Goethe University Frankfurt, the study examines the opportunities and occupational risks associated with increasingly autonomous healthcare AI.

Based on interviews with 25 physicians and nurses in Germany, the research identifies a central tension between automation and professional responsibility. Participants expected AI systems to reduce documentation, scheduling and coordination duties, potentially creating more time for patient care. However, they also anticipated new demands involving continuous monitoring, error detection, verification and intervention.

The findings suggest that greater technological autonomy does not necessarily translate into lower workloads or greater professional freedom. Instead, healthcare organizations may face a fundamental restructuring of clinical work in which operational tasks shift to machines while accountability, oversight and the consequences of failure remain closely tied to human professionals.

AI Is Changing Who Organizes Healthcare

Healthcare systems face mounting pressure from aging populations, increasing chronic disease burdens, workforce shortages and rising expenditure. Clinicians must navigate these challenges while managing extensive documentation requirements, fragmented care pathways and administrative processes that compete with time devoted to patients.

AI has increasingly been deployed to support medical image interpretation, risk prediction, clinical decision-making and documentation. Most conventional applications remain dependent on human initiation or supervision, with clinicians responsible for interpreting outputs and determining appropriate actions.

Agentic AI introduces a substantially different model. Rather than responding only to individual requests, these systems are designed to plan, coordinate and execute sequences of activities across digital environments. Potential applications include appointment scheduling, referral coordination, discharge management, patient follow-up, documentation and communication between healthcare services.

This changes the relationship between clinicians and technology. A conventional AI application may recommend an action, while an agentic system could initiate or coordinate several operational steps toward a broader objective. Such capabilities introduce questions about control, responsibility and the conditions under which autonomous activity should require human approval.

To investigate these concerns, the researchers interviewed 11 physicians and 14 nurses between November 2025 and February 2026. Participants worked across university hospitals, community hospitals and outpatient healthcare settings, representing different ages, professional experiences and levels of familiarity with digital technologies.

The researchers used semi-structured interviews and reflexive thematic analysis to identify patterns in participants' expectations. Five themes emerged, covering the transition from decision support to workflow organization, redistribution of work, professional accountability, occupational well-being and the organizational requirements for meaningful human oversight.

Participants generally considered administrative tasks more suitable for autonomous execution than clinical activities involving uncertainty, substantial risk or professional judgment. Their responses indicated that acceptance of AI autonomy depends not simply on what systems can perform, but on the consequences of allowing them to act independently.

The researchers characterize this changing relationship as delegated agency, describing how operational activities may transfer to AI while professionals retain varying degrees of authority, supervision and responsibility.

The Automation Paradox: Less Paperwork Could Mean More Mental Work

Administrative relief emerged as a prominent anticipated benefit of agentic AI. Physicians and nurses expected automated documentation, scheduling, referrals and communication to reduce repetitive work and potentially increase time available for direct patient care.

For nursing professionals, the prospect of spending more time with patients was particularly important. Physicians also associated reduced routine activity with greater opportunities to concentrate on meaningful clinical responsibilities.

However, the interviews revealed that eliminating manual tasks does not necessarily eliminate the work associated with those tasks. Autonomous systems still require monitoring, verification, correction and intervention when processes deviate from expected conditions.

The researchers describe this additional responsibility as second-order workload, referring to work generated by supervising automated processes rather than performing the original activities directly. Clinicians may no longer prepare every document or coordinate every appointment, but they could become responsible for checking whether those activities were completed appropriately.

As autonomous systems operate across multiple workflows, supervisory demands may become increasingly complex. Participants anticipated having to monitor several activities simultaneously, identify errors, assess unusual situations and determine when intervention was necessary. Such responsibilities could create fragmented attention and additional cognitive strain, particularly when systems behave unpredictably or require repeated verification. Poorly integrated technologies might consequently offset expected efficiency improvements by generating new forms of work.

The study identifies an important distinction between reducing the volume of tasks and reducing the overall burden experienced by healthcare professionals. Automation may remove repetitive administrative activities while increasing mental demands associated with supervision and uncertainty.

These anticipated changes also complicate assessments of occupational well-being. Participants expected potential improvements in time pressure, job satisfaction and opportunities for patient interaction, but simultaneously expressed concerns about stress, cognitive overload and difficulties disengaging from continuously operating systems.

One recurring concern involved AI processes continuing in the background even after professionals had completed their immediate duties. Persistent supervisory responsibility could make it harder to experience clear boundaries between task completion and ongoing monitoring.

The researchers caution against evaluating agentic AI exclusively through productivity gains, completed tasks or time saved. Its occupational consequences may depend on whether the benefits of automation outweigh the additional cognitive and supervisory responsibilities it introduces.

Importantly, the study did not measure actual changes in burnout, workload or professional well-being following deployment. Its findings reflect healthcare workers' expectations of emerging technologies rather than demonstrated occupational outcomes.

The Accountability Gap: When Machines Act but Clinicians Remain Responsible

Physicians and nurses anticipated that AI systems could initiate activities or coordinate clinical processes while healthcare professionals remained responsible for detecting mistakes, managing their consequences and responding to affected patients.

Physicians particularly emphasized accountability for adverse clinical outcomes, while nurses focused on the practical responsibility of recognizing inappropriate AI-supported actions and intervening during patient care.

The resulting imbalance creates what the researchers describe as a potential accountability gap. Operational authority may increasingly be delegated to autonomous systems without an equivalent transfer of professional responsibility.

Participants also expressed concerns about professional autonomy. Some anticipated that greater reliance on AI could gradually weaken their influence over clinical decisions and the organization of daily work. Others considered the transition an evolution of professional responsibilities toward supervision, validation and management of exceptional situations.

The study suggests that these outcomes depend on how organizations define decision-making authority, accountability and opportunities for human intervention. Greater technical autonomy does not automatically require reduced human agency, provided professionals retain meaningful control over system behavior.

However, retaining responsibility without sufficient visibility into automated actions presents a significant governance challenge. Clinicians cannot effectively supervise processes unless they understand what systems have done, which activities are underway and when intervention becomes necessary.

Transparency, traceability and explainability therefore emerged as essential requirements. Participants wanted the ability to examine AI behavior, challenge inappropriate actions and override systems when professional judgment required it. They also warned about automation bias, where repeated exposure to apparently reliable technology may encourage excessive trust and reduce critical scrutiny.

The researchers connect these concerns to emerging governance frameworks, including the European Union's AI Act and World Health Organization guidance concerning transparency, accountability and human oversight. They caution, however, that not every healthcare application of agentic AI automatically falls within high-risk regulatory classifications.

Effective governance consequently requires more than formally assigning a human supervisor. Healthcare organizations must establish clear boundaries for autonomous activity, define approval requirements, create escalation procedures and ensure professionals possess the training and authority necessary to intervene.

Future of Healthcare AI Depends on Workforce Protection

The research challenges technology-centered approaches that assess healthcare AI primarily through technical performance and operational efficiency. Its findings suggest that agentic AI should also be understood as an intervention affecting professional roles, organizational structures and working conditions.

Healthcare institutions considering autonomous systems may need to redesign workflows around supervision, exception management and clearly distributed responsibilities. Training must extend beyond basic software proficiency to include understanding system limitations, evaluating automated behavior and recognizing situations requiring intervention.

The researchers also emphasize involving healthcare professionals in system selection, design, testing and implementation. Such participation could help ensure that automated workflows reflect clinical realities and do not introduce unnecessary operational complexity.

For policymakers and healthcare administrators, the study identifies a need to evaluate workforce outcomes alongside patient safety and efficiency. Cognitive workload, professional autonomy, job satisfaction, burnout and the practical demands of supervision should form part of assessments of autonomous healthcare technologies.

These considerations are particularly relevant to healthcare systems experiencing staffing constraints, where automation may be viewed as a way to expand operational capacity. Yet the anticipated benefits could be undermined if reduced administrative work is replaced by excessive monitoring demands.

Several limitations remain important. The research draws on a qualitative sample of 25 German healthcare professionals, recruited partly through professional networks, and cannot represent the views of healthcare workers generally. Participants' expectations may also differ from their experiences following sustained use of autonomous systems.

Because agentic AI remains at an early stage of clinical implementation, the study does not establish its actual effects on patient outcomes, productivity, professional responsibility or occupational health. Future research must examine real-world deployments, measure workload and well-being over time, and investigate how different levels of autonomy affect clinical practice.