AI in precision oncology: Doctors don’t always trust AI - and that’s a good thing

Trust in AI is a crucial determinant of its integration into clinical practice. The study measured clinician trust levels and found a strong correlation between trust and agreement with AI recommendations. Clinicians who exhibited higher trust in AI were more likely to follow its suggestions, particularly when those suggestions aligned with their existing knowledge and clinical experience. However, trust was not static - it evolved based on AI’s perceived accuracy and its ability to justify its recommendations.

Artificial intelligence (AI) is increasingly becoming a critical tool in precision oncology, offering the potential to revolutionize treatment planning and execution. However, the integration of AI into clinical workflows remains a nuanced challenge, requiring a deep understanding of how clinicians interact with AI-generated recommendations.

A recent study, "Intricacies of Human–AI Interaction in Dynamic Decision-Making for Precision Oncology", published in Nat Commun 16, 1138 (2025, delves into the complexities of human-AI collaboration in response-adaptive radiotherapy for non-small cell lung cancer (NSCLC) and hepatocellular carcinoma (HCC). Conducted by Dipesh Niraula, Kyle C. Cuneo, Ivo D. Dinov, and their colleagues, the study uncovers significant insights into clinician behavior, decision-making processes, and the role of AI in shaping treatment strategies.

The complexity of human-AI decision-making

Human-AI interactions in precision oncology are not straightforward; they depend on a multitude of factors, including prior clinical knowledge, model transparency, and disease-specific considerations. The study distinguishes between model-agnostic and model-specific behaviors - where model-agnostic behavior reflects general trends in AI-assisted decision-making, while model-specific behavior pertains to clinician responses to the ARCliDS AI system used in this study.

ARCliDS, an AI-driven clinical decision support system, was employed to assist in knowledge-based response-adaptive radiotherapy (KBR-ART), a technique designed to dynamically adjust treatment based on real-time patient response. The study found that clinicians displayed varying levels of reliance on AI, with some critically evaluating its recommendations, others making minor adjustments, and a subset outright rejecting AI input due to skepticism about its validity. This underscores a broader issue in AI adoption - trust in AI systems is not uniformly distributed across medical professionals, and skepticism can arise from concerns about AI biases, lack of explainability, or deviations from established clinical protocols.

Interestingly, the study found that AI recommendations were more readily accepted when they aligned with traditional treatment approaches, but AI-generated suggestions that deviated from conventional methodologies were met with significant scrutiny. This highlights the challenge of integrating AI into areas of oncology that lack well-established treatment paradigms, where AI's predictive power could be particularly valuable but is simultaneously met with the greatest resistance.

AI's influence on clinical decisions

Through an analysis of 72 evaluations for NSCLC and 72 for HCC, the study measured how AI-assisted decisions compared to unassisted ones. AI influence was not homogenous across cases; decision adjustments were observed in approximately 57% of NSCLC evaluations and 47% of HCC evaluations. Notably, clinicians were more likely to modify their decisions if their initial choice significantly deviated from AI recommendations, indicating a corrective feedback loop wherein AI served as a secondary validation tool rather than a primary decision-maker.

The study also observed distinct trends based on treatment modality. In NSCLC cases, AI recommendations tended to encourage dose escalation or de-escalation based on tumor response dynamics. However, in HCC cases, AI was more focused on minimizing toxicity risks, as liver function plays a critical role in treatment tolerability. This disease-specific variation in AI's decision-making highlights the need for AI models to be fine-tuned for different cancer types rather than assuming a one-size-fits-all approach.

Moreover, clinician confidence in AI recommendations was a key variable influencing decision alignment. Clinicians reported higher confidence levels when AI recommendations were accompanied by transparent explanations and outcome predictions. The study suggests that AI adoption in oncology will require robust explainability frameworks that provide clinicians with clear reasoning for AI-generated decisions, rather than black-box predictions that lack interpretability.

Trust in AI and the future of human-AI collaboration

Trust in AI is a crucial determinant of its integration into clinical practice. The study measured clinician trust levels and found a strong correlation between trust and agreement with AI recommendations. Clinicians who exhibited higher trust in AI were more likely to follow its suggestions, particularly when those suggestions aligned with their existing knowledge and clinical experience. However, trust was not static - it evolved based on AI's perceived accuracy and its ability to justify its recommendations.

One of the study's most striking findings was AI's potential to reduce inter-physician variability. When AI-assisted decision-making was employed, evaluator agreement on treatment recommendations increased, suggesting that AI could play a crucial role in standardizing treatment approaches and minimizing discrepancies in clinical decision-making. However, this homogenization of decision-making raises ethical and clinical concerns - should AI serve merely as a guide, or should it have a more authoritative role in defining treatment protocols? The study suggests that a balanced approach is necessary, where AI is leveraged for its analytical strengths but final decisions remain in the hands of experienced clinicians who can account for nuances AI may overlook.