Seven out of 100 words in speech recognition-generated documents contain errors, many of which involve clinical information, according to the authors of a recent study.1 “Some errors could potentially lead to malpractice,” says Foster Goss, DO, one of the study’s authors.
Previously, Goss worked with other researchers on a study of the ED setting specifically. They analyzed a random sample of 100 notes dictated by attending EPs using speech recognition software in the first half of 2012.2 Researchers found 1.3 errors per note. Up to 15% of errors were found to be critical, potentially leading to miscommunication that could affect patient care.
“Most clinicians perceive the error rates to be lower,” says Goss, adding that, unfortunately, “errors such as these, or simple syntax errors, are often left uncorrected in the ED chart.” Two recent cases serve as examples:
- A 25-year-old female presented to the ED with a soft tissue infection on her arm. The EP’s comment that the patient had missed her “period” was interpreted by the software as a punctuation mark. She returned to the ED the next day with worsening cellulitis on her arm and was prescribed an antibiotic that was contraindicated during pregnancy.
- A physician’s dictation of “inadequate evaluation to exclude neoplasia” was interpreted as “adequate evaluation to exclude neoplasia.”
It is not that EPs are careless. “They genuinely don’t want their notes to have errors. But this is a sign of a more systemic problem,” says Goss, an assistant professor of emergency medicine at the University of Colorado School of Medicine. Asked to see more patients in less time, EPs turn to speech recognition to improve efficiency. “Errors may be the byproduct of trying to comply with time constraints and documentation requirements,” Goss offers.
For ED providers, speech recognition technology “is incredibly valuable in our clinical workflows, allowing us to tell the patient story and narrative,” Goss says. Post-processing tools are in development to detect mistakes. Until then, correction will rely on careful proofreading of dictations by the EP. “Like any technology, it’s not perfect,” Goss says. “It makes mistakes. As a result, clinicians need to carefully proofread their dictations.”
Of the raw speech recognition transcriptions evaluated in the 2018 study, 96.3% contained at least one error, and 63.6% contained at least one clinically significant error.1 “Our findings confirmed the need for manual review in this process,” says Li Zhou, MD, PhD, lead author. “User training, quality assurance, and auditing may also be helpful.”
The error rate decreased significantly after a medical transcriptionist review. The rate declined further after the clinician reviewed the edited transcript. “The comparatively low error rate in edited and signed notes highlights the crucial role that manual review plays,” says Zhou, an associate professor at the division of general internal medicine and primary care at Brigham and Women’s Hospital in Boston.
A recent analysis of 51,000 closed malpractice claims from 2014 found nine cases in which inaccurately transcribed medical dictations by speech recognition systems led to patient harm.3
“Very little is known about speech recognition errors in the ED setting,” notes Max Topaz, PhD, RN, MA, lead author. Although speech recognition in the ED was found to be a contributing factor in a very small fraction of the cases, the researchers found that the number of cases has increased over time.
“We found that speech recognition technology contributed to the cases rather than directly causing the patient harm,” says Zhou, who worked with Topaz and others on this research.
Sloppy charting was an issue in additional malpractice cases included in the analysis.
“We couldn’t be certain, though, to what extent speech recognition errors were related to careless documentation,” says Topaz, the Elizabeth Standish Gill associate professor of nursing at Columbia University School of Nursing in New York City.
Regardless of the precise number of cases, there is no question that speech recognition technology is used increasingly in EDs. Considering this, says Topaz, “we expect to see more issues related to this technology in the near future.”
REFERENCES
- Zhou L, Blackley SV, Kowalski L, et al. Analysis of errors in dictated clinical documents assisted by speech recognition software and professional transcriptionists analysis. JAMA Netw Open 2018;1. pii: e180530. doi:10.1001/jamanetworkopen.2018.0530. Epub 2018 Jul 6.
- Goss FR, Zhou L, Weiner SG. Incidence of speech recognition errors in the emergency department. Int J Med Inform 2016;93:70-73.
- Topaz M, Schaffer A, Lai KH, et al. Medical malpractice trends: Errors in automated speech recognition. J Med Syst 2018;42:153.
Seven out of 100 words in speech recognition-generated documents contain errors, many of which involve clinical information, according to the authors of a recent study.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content