AI-Powered Health: GutGPT's Impact on Gastrointestinal Bleeding Risk Assessment

CoE-EDP, VisionRICoE-EDP, VisionRI | Updated: 13-06-2024 14:50 IST | Created: 13-06-2024 14:50 IST
AI-Powered Health: GutGPT's Impact on Gastrointestinal Bleeding Risk Assessment
Representative Image

The study by various researchers from Yale University, USA, CUNY Baruch College, USA, Yale School of Medicine, USA, and University of California, Berkeley, USA explores the potential of large language models (LLMs) like GutGPT to enhance clinical decision support through conversational interfaces. GutGPT was developed to predict and manage gastrointestinal (GI) bleeding risk. To evaluate its effectiveness, the researchers conducted clinical simulation scenarios involving emergency medicine physicians, internal medicine physicians, and medical students, aiming to assess how GutGPT impacts physician acceptance, trust in AI clinical decision support systems (AI-CDSS), and their mastery of GI bleeding management.

The Promise and Challenges of LLMs in Healthcare

Large language models, such as OpenAI’s GPT-4, offer advanced capabilities for clinical decision support by providing a conversational interface for on-demand information retrieval and summarization. ChatGPT, a well-known LLM, has shown high accuracy and consistency in explaining medical questions, making it a promising tool for medical training. However, the interaction between clinicians and LLMs is not well understood, posing challenges for clinical implementation. Key concerns include trust, understanding AI reasoning, legal liability, and potential workflow disruptions. Epic Systems, a leading electronic health record (EHR) vendor, is integrating OpenAI’s LLMs into its platform to boost healthcare provider productivity through automation and enhanced decision support.

Methodology: Simulating Real-World Scenarios

The Yale University team developed a machine learning (ML) model to predict GI bleeding risk and integrated it into an interactive dashboard within the EHR. GutGPT, an AI chatbot, interfaces with this ML model to provide risk predictions and evidence-based recommendations for managing acute upper GI bleeding. The study was conducted in a simulation center, providing controlled conditions for evaluating clinician attitudes and the usability of GutGPT. Participants were randomized into two groups: one using GutGPT with the interactive dashboard, and the other using the dashboard alone. Surveys measured trust, acceptability, intention to use, and usability before and after the simulation scenarios. Preliminary results showed mixed effects on acceptance but suggested that GutGPT improved participants' content mastery.

Findings: Trust and Knowledge Improvement

Preliminary findings from 55 participants indicated that both groups showed increased intention to use AI-CDSS and trust after the simulations. The perceived ease of use, or effort expectancy, particularly increased for those using GutGPT. Participants also demonstrated improved knowledge in managing GI bleeding, with GutGPT users finding the system helpful for patient triage. However, there were concerns that AI systems might not fully consider the social, emotional, and physical nuances of clinical decision-making.

Future Directions: Refining and Optimizing AI-CDSS

The study underscores the potential of using medical simulations to evaluate LLM-based chatbots like GutGPT for clinical decision support. Simulations provide a safe, controlled environment for testing new technologies and understanding clinician attitudes without compromising patient safety. The results suggest that GutGPT and interactive dashboards can enhance knowledge acquisition and maintain trust in simulated scenarios, although their impact on trust and acceptance is mixed. Future directions for this research include comparing different LLMs, optimizing settings to minimize incorrect responses (hallucinations), and refining LLM architectures for better clinical relevance. The use of virtual and augmented reality could also improve the accessibility, scalability, and customization of future simulation studies. The study acknowledges the technical support from the Yale Center for Healthcare Simulation and continues to refine and optimize AI-CDSS tools for clinical use.

This study demonstrates the potential of GutGPT to support clinical decision-making in managing GI bleeding. While there are challenges to address, such as trust and the nuanced understanding of patient care, the preliminary results are promising. With continued research and refinement, AI-CDSS like GutGPT could become valuable tools in clinical practice, enhancing decision-making and improving patient outcomes.

  • Devdiscourse
Give Feedback