AI and Ethics: A Systematic Review of the Ethical Considerations of Large Language Model Use in Surgery Research
Sophia M. Pressman, Sahar Borna, Cesar A. Gomez-Cabello, Syed A. Haider, Clifton Haider, Antonio J. Forte
Healthcare, 13 April 2024; 12(8)
Abstract
Introduction
As large language models receive greater attention in medical research, the investigation of ethical considerations is warranted. This review aims to explore surgery literature to identify ethical concerns surrounding these artificial intelligence models and evaluate how autonomy, beneficence, nonmaleficence, and justice are represented within these ethical discussions to provide insights in order to guide further research and practice.
Methods
A systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Five electronic databases were searched in October 2023. Eligible studies included surgery-related articles that focused on large language models and contained adequate ethical discussion. Study details, including specialty and ethical concerns, were collected.
Results
The literature search yielded 1179 articles, with 53 meeting the inclusion criteria. Plastic surgery, orthopedic surgery, and neurosurgery were the most represented surgical specialties. Autonomy was the most explicitly cited ethical principle. The most frequently discussed ethical concern was accuracy (n = 45, 84.9%), followed by bias, patient confidentiality, and responsibility.
Conclusion
The ethical implications of using large language models in surgery are complex and evolving. The integration of these models into surgery necessitates continuous ethical discourse to ensure responsible and ethical use, balancing technological advancement with human dignity and safety.
Category: Artificial Intelligence
Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial
Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial
Bilgin Osmanodja, Zeineb Sassi, Sascha Eickmann, Carla Maria Hansen, Roland Roller, Aljoscha Burchardt, David Samhammer, Peter Dabrock, Sebastian Möller, Klemens Budde, Anne Herrmann
JMIR Research Protocols, 1 April 2024
Abstract
Background
Patients after kidney transplantation eventually face the risk of graft loss with the concomitant need for dialysis or retransplantation. Choosing the right kidney replacement therapy after graft loss is an important preference-sensitive decision for kidney transplant recipients. However, the rate of conversations about treatment options after kidney graft loss has been shown to be as low as 13% in previous studies. It is unknown whether the implementation of artificial intelligence (AI)–based risk prediction models can increase the number of conversations about treatment options after graft loss and how this might influence the associated shared decision-making (SDM).
Objective
This study aims to explore the impact of AI-based risk prediction for the risk of graft loss on the frequency of conversations about the treatment options after graft loss, as well as the associated SDM process.
Methods
This is a 2-year, prospective, randomized, 2-armed, parallel-group, single-center trial in a German kidney transplant center. All patients will receive the same routine post–kidney transplant care that usually includes follow-up visits every 3 months at the kidney transplant center. For patients in the intervention arm, physicians will be assisted by a validated and previously published AI-based risk prediction system that estimates the risk for graft loss in the next year, starting from 3 months after randomization until 24 months after randomization. The study population will consist of 122 kidney transplant recipients >12 months after transplantation, who are at least 18 years of age, are able to communicate in German, and have an estimated glomerular filtration rate <30 mL/min/1.73 m2. Patients with multi-organ transplantation, or who are not able to communicate in German, as well as underage patients, cannot participate. For the primary end point, the proportion of patients who have had a conversation about their treatment options after graft loss is compared at 12 months after randomization. Additionally, 2 different assessment tools for SDM, the CollaboRATE mean score and the Control Preference Scale, are compared between the 2 groups at 12 months and 24 months after randomization. Furthermore, recordings of patient-physician conversations, as well as semistructured interviews with patients, support persons, and physicians, are performed to support the quantitative results.
Results
The enrollment for the study is ongoing. The first results are expected to be submitted for publication in 2025.
Conclusions
This is the first study to examine the influence of AI-based risk prediction on physician-patient interaction in the context of kidney transplantation. We use a mixed methods approach by combining a randomized design with a simple quantitative end point (frequency of conversations), different quantitative measurements for SDM, and several qualitative research methods (eg, records of physician-patient conversations and semistructured interviews) to examine the implementation of AI-based risk prediction in the clinic.
Monitoring Mental Health: Legal and Ethical Considerations of Using Artificial Intelligence in Psychiatric Wards
Monitoring Mental Health: Legal and Ethical Considerations of Using Artificial Intelligence in Psychiatric Wards
Barry Solaiman, Abeer Malik, Suhaila Ghuloum
American Journal Of Law & Medicine, 12 February 2024
Abstract
Artificial intelligence (AI) is being tested and deployed in major hospitals to monitor patients, leading to improved health outcomes, lower costs, and time savings. This uptake is in its infancy, with new applications being considered. In this Article, the challenges of deploying AI in mental health wards are examined by reference to AI surveillance systems, suicide prediction and hospital administration. The examination highlights risks surrounding patient privacy, informed consent, and data considerations. Overall, these risks indicate that AI should only be used in a psychiatric ward after careful deliberation, caution, and ongoing reappraisal.
Shaping the future of AI in healthcare through ethics and governance
Shaping the future of AI in healthcare through ethics and governance
Rabaï Bouderhem
Humanities and Social Sciences Communications, 15 March 2024
Open Access
Abstract
The purpose of this research is to identify and evaluate the technical, ethical and regulatory challenges related to the use of Artificial Intelligence (AI) in healthcare. The potential applications of AI in healthcare seem limitless and vary in their nature and scope, ranging from privacy, research, informed consent, patient autonomy, accountability, health equity, fairness, AI-based diagnostic algorithms to care management through automation for specific manual activities to reduce paperwork and human error. The main challenges faced by states in regulating the use of AI in healthcare were identified, especially the legal voids and complexities for adequate regulation and better transparency. A few recommendations were made to protect health data, mitigate risks and regulate more efficiently the use of AI in healthcare through international cooperation and the adoption of harmonized standards under the World Health Organization (WHO) in line with its constitutional mandate to regulate digital and public health. European Union (EU) law can serve as a model and guidance for the WHO for a reform of the International Health Regulations (IHR).
Consent and Identifiability for Patient Images in Research, Education, and Image-Based Artificial Intelligence
Consent and Identifiability for Patient Images in Research, Education, and Image-Based Artificial Intelligence
Research Letter
Trina Salvador, Lilly Gu, Jennifer L. Hay, Nicholas R. Kurtansky, Ruth Masterson-Creber, Allan C. Halpern, Veronica Rotemberg
JAMA Dermatology, 13 March 2024
Abstract
Increasing use of imaging for research, education, and development of image-based artificial intelligence (AI) is parallel to increasing concerns about confidentiality and autonomy. Regulatory requirements for collecting and processing personal information vary geographically, but even the most stringent legal guidelines do not require informed consent for sharing of deidentified data. Despite widespread use of imaging in dermatology and portability of digital image formats that enable both rapid intentional and inadvertent image sharing, information about attitudes and preferences on consent and identifiability are limited. Consequently, processes for obtaining informed consent are not standardized across clinical practices and research journals.4 To inform practices for protecting privacy, we performed a survey study to elucidate perspectives on image use, consent, and identifiability.
An In-Depth Qualitative Interview: The Impact of Artificial Intelligence (AI) on Consent and Transparency
An In-Depth Qualitative Interview: The Impact of Artificial Intelligence (AI) on Consent and Transparency
Book Chapter
Sharon L. Burton, Darrell N. Burrell, Calvin Nobles, Yoshino W. White, Maurice E. Dawson, Kim L. Brown-Jackson, S Rachid Muller, Dustin I. Bessette
Multisector Insights in Healthcare, Social Sciences, Society, and Technology, 2024 [IGI Global]
Abstract
AI is impacting consent and transparency adversely. Although AI can potentially augment transparency in decision-making via advanced technology, it is creating new concerns. This chapter focuses on the impact of AI systems on individuals’ ability to provide informed consent for using their data, and the relationship between transparency in AI decision-making processes and issues related to accountability and trust. Discussed are GDPR (European Union General Data Protection Regulation), and CCPA (California Consumer Privacy Act) due to their consent and transparency within their broader privacy protection frameworks. Applied is a qualitative methodology and in-depth interview design using a communication and collaboration platform to explain the connection between AI consent and transparency. Research results offer avenues to understanding the challenges of informed consent and legal and ethical considerations regarding consent and transparency. Beneficiaries of this research are practitioners, academics, and learners in AI, cybersecurity, and criminology/criminal justice.
Spotlight Articles
Recognizing the emergence of large language models (LLMs) and generative AI, our spotlight section this month focuses on articles which appeared the Journal of Medical Ethics, published online on January 23rd 2024. The articles address the use of LLMs and generative AI in informed consent procedures and, more broadly, the use of this technology within the medical ethics space.
In the editorial by Zohny et al, Generative AI and medical ethics: the state of play, the authors provide an overview of how LLMs are being used in medical ethics currently, and note that the technology lacks the maturity for nuanced ethical decision making at this time.
Allen et al. address the potential for LLMs to be used to facilitate surgical consent transactions with patients in Consent-GPT: is it ethical to delegate procedural consent to conversational AI? The authors raise several concerns with this practice, including the risk of misinformation, the absence of trust that one might have in the doctor-patient relationship, the potential for ‘click-through’ consent rather than fulsome consent, and the lack of clarity surrounding who has responsibility for an LLM informed consent transaction.
In Assessing the performance of ChatGPT in bioethics: a large language model’s moral compass in medicine, Chen et al. assess that LLMs have the potential to address certain aspects of medical ethics that required social intelligence but struggled in nuanced areas such as informed consent transactions.
Finally, Balas et al. found in Exploring the potential utility of AI large language models for medical ethics: an expert panel evaluation of GPT- 4 that when faced with ethical decision making, LLMs were able to articulate the principled issues at hand but without much understanding or depth to what each issue might translate to in terms of patient experience LLMs tested were also unable to integrate ethical and legal concepts in a satisfactory manner. The authors believe that at the moment AI may be used to compliment, but not replace, healthcare practitioner involvement in informed consent transactions.
Generative AI and medical ethics: the state of play
Editorial
Hazem Zohny, Sebastian Porsdam Mann, Brian D Earp, John McMillan
Journal of Medical Ethics, 23 January 2024
Excerpt
Since their public launch, a little over a year ago, large language models (LLMs) have inspired a flurry of analysis about what their implications might be for medical ethics, and for society more broadly. Much of the recent debate has moved beyond categorical evaluations of the permissibility or impermissibility of LLM use in different general contexts (eg, at work or school), to more fine-grained discussions of the criteria that should govern their appropriate use in specific domains or towards certain ends. With each passing week, it seems more and more inevitable that LLMs will be a pervasive feature of many, if not most, of our lives. It would not be possible—and would not be desirable—to prohibit them across the board. We need to learn how to live with LLMs; to identify and mitigate the risks they pose to us, to our fellow creatures, and the environment; and to harness and guide their powers to better ends. This will require thoughtful regulation, sustained cooperation across nations, cultures and fields of inquiry; and all of this must be grounded in good ethics…
Consent-GPT: is it ethical to delegate procedural consent to conversational AI?
Current Controversy
Jemima Winifred Allen, Brian D Earp, Julian Koplin, Dominic Wilkinson
Journal of Medical Ethics, 23 January 2024
Abstract
Obtaining informed consent from patients prior to a medical or surgical procedure is a fundamental part of safe and ethical clinical practice. Currently, it is routine for a significant part of the consent process to be delegated to members of the clinical team not performing the procedure (eg, junior doctors). However, it is common for consent-taking delegates to lack sufficient time and clinical knowledge to adequately promote patient autonomy and informed decision-making. Such problems might be addressed in a number of ways. One possible solution to this clinical dilemma is through the use of conversational artificial intelligence using large language models (LLMs). There is considerable interest in the potential benefits of such models in medicine. For delegated procedural consent, LLM could improve patients’ access to the relevant procedural information and therefore enhance informed decision-making.
In this paper, we first outline a hypothetical example of delegation of consent to LLMs prior to surgery. We then discuss existing clinical guidelines for consent delegation and some of the ways in which current practice may fail to meet the ethical purposes of informed consent. We outline and discuss the ethical implications of delegating consent to LLMs in medicine concluding that at least in certain clinical situations, the benefits of LLMs potentially far outweigh those of current practices.
Assessing the performance of ChatGPT in bioethics: a large language model’s moral compass in medicine
Original research
Jamie Chen, Angelo Cadiente, Lora J Kasselman, Bryan Pilkington
Journal of Medical Ethics, 23 January 2024
Abstract
Chat Generative Pre-Trained Transformer (ChatGPT) has been a growing point of interest in medical education yet has not been assessed in the field of bioethics. This study evaluated the accuracy of ChatGPT-3.5 (April 2023 version) in answering text-based, multiple choice bioethics questions at the level of US third-year and fourth-year medical students. A total of 114 bioethical questions were identified from the widely utilised question banks UWorld and AMBOSS. Accuracy, bioethical categories, difficulty levels, specialty data, error analysis and character count were analysed. We found that ChatGPT had an accuracy of 59.6%, with greater accuracy in topics surrounding death and patient–physician relationships and performed poorly on questions pertaining to informed consent. Of all the specialties, it performed best in paediatrics. Yet, certain specialties and bioethical categories were under-represented. Among the errors made, it tended towards content errors and application errors. There were no significant associations between character count and accuracy. Nevertheless, this investigation contributes to the ongoing dialogue on artificial intelligence’s (AI) role in healthcare and medical education, advocating for further research to fully understand AI systems’ capabilities and constraints in the nuanced field of medical bioethics.
Exploring the potential utility of AI large language models for medical ethics: an expert panel evaluation of GPT-4
Original Research
Michael Balas, Jordan Joseph Wadden, Philip C Hébert, Eric Mathison, Marika D Warren, Victoria Seavilleklein, Daniel Wyzynski, Alison Callahan, Sean A Crawford, Parnian Arjmand, Edsel B Ing
Journal of Medical Ethics, 23 January 2024
Abstract
Integrating large language models (LLMs) like GPT-4 into medical ethics is a novel concept, and understanding the effectiveness of these models in aiding ethicists with decision-making can have significant implications for the healthcare sector. Thus, the objective of this study was to evaluate the performance of GPT-4 in responding to complex medical ethical vignettes and to gauge its utility and limitations for aiding medical ethicists. Using a mixed-methods, cross-sectional survey approach, a panel of six ethicists assessed LLM-generated responses to eight ethical vignettes.
The main outcomes measured were relevance, reasoning, depth, technical and non-technical clarity, as well as acceptability of GPT-4’s responses. The readability of the responses was also assessed. Of the six metrics evaluating the effectiveness of GPT-4’s responses, the overall mean score was 4.1/5. GPT-4 was rated highest in providing technical (4.7/5) and non-technical clarity (4.4/5), whereas the lowest rated metrics were depth (3.8/5) and acceptability (3.8/5). There was poor-to-moderate inter-rater reliability characterised by an intraclass coefficient of 0.54 (95% CI: 0.30 to 0.71). Based on panellist feedback, GPT-4 was able to identify and articulate key ethical issues but struggled to appreciate the nuanced aspects of ethical dilemmas and misapplied certain moral principles.
This study reveals limitations in the ability of GPT-4 to appreciate the depth and nuanced acceptability of real-world ethical dilemmas, particularly those that require a thorough understanding of relational complexities and context-specific values. Ongoing evaluation of LLM capabilities within medical ethics remains paramount, and further refinement is needed before it can be used effectively in clinical settings.
Meta-Health. Using The Metaverse To Facilitate The Understanding Of The Patient’s Informed Consent And Their Perioperative Process
Meta-Health. Using The Metaverse To Facilitate The Understanding Of The Patient’s Informed Consent And Their Perioperative Process
L Sánchez-Guillén, C Rebollo-Santamaría, N Montaña-Miranda, M Pérez-Berenguer, P Martínez-Galisteo, C Lillo-García, F López-Rodríguez-Arias, A Arroyo
British Journal of Surgery, 9 February 2024; supplement 1
Abstract
Introduction
The informed consent (IC) process can sometimes result in a lack of understanding due to the technical nature of the information provided. Coupled with the physical disorientation within the hospital, this can lead to perioperative anxiety and a greater risk of complications.
Methods
META-health is an application for mobile devices that seeks to improve these deficiencies using metaverse and extended reality visualization technologies. By recreating the hospital environment in a virtual setting, it allows repeating the preoperative consultation and visualization of various hospital areas, including the hospitalization floor, operating room, and consultation spaces. Multimedia content is included so that the information can be more easily understood. In addition, gamification has been implemented as an option to integrate entertainment into the understanding process of stomatal management.
Results
The pilot application was modeled and developed, and the first focus group was held to assess the quality, management, orientation, and comprehension of content. The sample of 12 patients indicated proposals for improvement and functional errors: 66.7% required control of the explanatory videos, 75% requested information on ostomies and difficulties in handling the joystick. The application is currently focused on colorectal cancer patients and intends to expand to other conditions and functionalities to become a functional metaverse. Finally, 100% of participants confirmed the usefulness of the idea and the importance of receiving information and support throughout the process.
Conclusion
The use of extended reality and the metaverse improves the understanding of informed consent (IC) by patients and relatives, as well as decreases perioperative anxiety.
Autonomy and Informed Consent: Ensuring patients and their families are well-informed about AI-assisted decisions
Autonomy and Informed Consent: Ensuring patients and their families are well-informed about AI-assisted decisions
Elisha Blessing, Kaledio Potter, Hubert Klaus
Research Gate, 1 February 2024
Abstract
The integration of Artificial Intelligence (AI) into healthcare presents significant opportunities for improving patient outcomes, personalizing treatments, and enhancing diagnostic accuracy. However, this technological advancement also raises crucial ethical considerations, particularly concerning patient autonomy and informed consent. As AI-assisted decision-making becomes more prevalent in healthcare settings, ensuring that patients and their families are well-informed about the nature, benefits, and risks of AI interventions is paramount. This paper explores the challenges and strategies associated with maintaining autonomy and securing informed consent in the era of AI-assisted healthcare. We delve into the significance of patient autonomy as a fundamental principle of medical ethics and the complexities introduced by AI technologies that may challenge this autonomy. The paper highlights the crucial role of healthcare providers in supporting informed decision-making by patients, emphasizing the need for clear communication about AI’s capabilities and limitations. Further, we address the traditional principles of informed consent and how they are complicated by the integration of AI in healthcare. These complexities include the difficulty of explaining AI’s decision-making processes and ensuring patients understand the implications of AI-assisted treatments. We propose strategies for enhancing patient and family understanding of AI-assisted decisions, including educational programs, training for healthcare professionals, and the development of patient-centered AI explanations.
The paper also reviews case studies and examples of successful implementations of AI in healthcare, providing insights into best practices and the impact on patient satisfaction and trust. Additionally, we examine the ethical and legal frameworks governing AI in healthcare, identifying the need for updated policies to address AI-specific issues and international perspectives on autonomy and informed consent. Challenges such as biases in AI algorithms, privacy and security of patient data, and overcoming skepticism and fear of technology are discussed, alongside future directions anticipated in AI development that may impact patient care and consent processes.
The paper concludes with a call to action for ongoing dialogue, research, and policy development to ensure that the ethical principles of autonomy and informed consent are upheld in the rapidly evolving landscape of AI in healthcare. This exploration underscores the importance of ensuring that patients and their families are adequately informed, empowering them to make decisions that align with their values and preferences in the context of AI-assisted healthcare.
Telehealth and AI: An Ethical Examination of Remote Healthcare Services and the Implications for Patient Care and Privacy
Telehealth and AI: An Ethical Examination of Remote Healthcare Services and the Implications for Patient Care and Privacy
Andi Saputra, Siti Aminah
Quarterly Journal of Computational Technologies for Healthcare, 6 January 2024
Abstract
Background
The integration of artificial intelligence (AI) in telehealth has revolutionized healthcare delivery, offering unprecedented opportunities for remote diagnosis, treatment, and patient monitoring. This research aims to critically examine the ethical implications of this technological convergence.
Objective
To explore the ethical dimensions of AI-enhanced telehealth, focusing on accessibility, quality of care, patient privacy, data security, informed consent, regulatory challenges, and the long-term societal impacts.
Methods
The study employs a comprehensive literature review and ethical analysis framework, examining current practices, patient outcomes, and regulatory policies related to AI in telehealth.
Results
The findings highlight the potential of AI-enhanced telehealth in increasing healthcare accessibility, especially in remote and underserved areas. However, challenges such as digital divide, data privacy concerns, and the risk of algorithmic bias are identified as key ethical issues. The lack of comprehensive regulatory frameworks and standards for AI in healthcare poses significant challenges in ensuring equitable and safe care. Furthermore, the study underscores the importance of informed consent in the context of AI-driven healthcare services.
Conclusion
While AI-enhanced telehealth offers significant benefits in healthcare delivery, it raises critical ethical concerns that must be addressed. Ensuring equitable access, safeguarding patient privacy, maintaining the quality of care, and developing robust regulatory frameworks are essential for the responsible integration of AI in telehealth services. Future research should focus on developing ethical guidelines and policies that keep pace with technological advancements in healthcare.
Center for Informed Consent Integrity 