Ethical Concerns with Large Language Models in Surgery

By Stacey Kusterbeck

Large language models (LLMs) in surgery have the potential to enhance decision-making, documentation, and patient engagement. However, the body of literature addressing the ethical concerns of applying LLMs in surgical settings is relatively limited. “Most existing research tends to focus on the technical performance, integration, and potential benefits of these technologies, rather than the ethical aspects of these tools,” observes Sahar Borna, MD, a research fellow in the Division of Plastic Surgery at Mayo Clinic in Jacksonville, FL.

Borna and colleagues conducted a review of the surgery literature on ethics of LLMs.¹ The researchers analyzed how the ethical principles of autonomy, beneficence, nonmaleficience, and justice were discussed in 53 studies. The aim was to understand, from an ethical perspective, the biggest concerns with LLMs in the surgical field. Some key findings:

Autonomy was the ethical principle explicitly cited most often.

“The complex nature of LLMs may complicate the informed consent process, potentially impacting patients’ ability to make well-informed decisions regarding their treatment,” says Sophia M. Pressman, BS, the study’s lead author and a research trainee at the Mayo Clinic in Jacksonville in the Division of Plastic Surgery.

Accuracy was the most frequently discussed ethical concern, cited in about 85% of studies.

“This underscores a significant apprehension within the surgical community regarding the potential ramifications of errors associated with LLMs,” says Pressman. Surgeons are concerned that errors in LLM outputs might lead to misdiagnoses or incorrect surgical procedures, for example. “Although LLMs should not be used to replace a physician’s clinical judgment, future applications may see the involvement of LLMs to automate certain processes. But this will depend upon accurate outputs and validated LLMs,” says Pressman.

Bias was frequently discussed, including the potential for perpetuation of existing healthcare disparities.

Patient confidentiality was cited often, based on the concern that breaches can undermine patient trust and privacy.

Overall, the study findings emphasize the need for robust ethical frameworks for responsible integration of LLMs into clinical practice, according to the authors.

“Surgeons are particularly concerned about the implications of LLMs for clinical decision-making. Inaccurate and biased information is a major concern, as this can greatly undermine the clinical decision-making process,” according to study author Antonio J. Forte, MD, PhD, director of the Plastic Surgery Outcomes Research Team at Mayo Clinic. To address ethical concerns with LLMs in surgery, Forte says that these steps are essential:

Having continuous ethical dialogue to address evolving challenges.

LLMs are not yet directly involved in performing surgeries. “As LLMs become more integrated into surgical settings, it’s crucial to address the ethical issues they might raise proactively,” says Forte. Evolving concerns include ensuring the accuracy of LLMs, managing bias, and maintaining patient confidentiality and autonomy.

Institutional-level ethical dialogue on LLMs in surgery could include multidisciplinary discussions and integration of ethicists into surgical teams to address ethical issues in real time. “Ethical guidelines and robust oversight mechanisms will be essential to ensure these technologies benefit patient care while upholding ethical standards,” adds Forte.

Improving the transparency and validity of LLMs in hospitals.

“Ethicists can guide the ethical integration and monitoring of LLMs, while IT experts can ensure their technical robustness and transparency,” suggests Forte.

Educating healthcare providers on ethical use of LLMs.

Ethicists can help to develop training modules or can offer workshops on the ethical use of AI in clinical practice.

Developing hospital-wide policies for the ethical integration of LLMs into healthcare practices.

“Ethicists can help ensure that these models are being utilized in a way that supports patient rights and improves the quality of healthcare,” says Forte.

REFERENCE

1. Pressman SM, Borna S, Gomez-Cabello CA, et al. AI and ethics: A systematic review of the ethical considerations of large language model use in surgery research. Healthcare (Basel). 2024;12(8):825.