Foster R Goss1, Li Zhou2, Scott G Weiner3. 1. University of Colorado, Department of Emergency Medicine, Aurora, CO, United States; Tufts Medical Center, Department of Emergency Medicine and Clinical Decision Making, Boston, MA, United States. Electronic address: Foster.Goss@ucdenver.edu. 2. Division of General Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA, United States; Clinical & Quality Analysis, Partners HealthCare System, Boston, MA, United States; Clinical Informatics, Partners eCare, Partners HealthCare System, Boston, MA, United States; Harvard Medical School, Boston, MA, United States. 3. Division of Health Policy Translation, Department of Emergency Medicine, Brigham and Women's Hospital, Boston, MA, United States.
Abstract
BACKGROUND: Physician use of computerized speech recognition (SR) technology has risen in recent years due to its ease of use and efficiency at the point of care. However, error rates between 10 and 23% have been observed, raising concern about the number of errors being entered into the permanent medical record, their impact on quality of care and medical liability that may arise. Our aim was to determine the incidence and types of SR errors introduced by this technology in the emergency department (ED). SETTING: Level 1 emergency department with 42,000 visits/year in a tertiary academic teaching hospital. METHODS: A random sample of 100 notes dictated by attending emergency physicians (EPs) using SR software was collected from the ED electronic health record between January and June 2012. Two board-certified EPs annotated the notes and conducted error analysis independently. An existing classification schema was adopted to classify errors into eight errors types. Critical errors deemed to potentially impact patient care were identified. RESULTS: There were 128 errors in total or 1.3 errors per note, and 14.8% (n=19) errors were judged to be critical. 71% of notes contained errors, and 15% contained one or more critical errors. Annunciation errors were the highest at 53.9% (n=69), followed by deletions at 18.0% (n=23) and added words at 11.7% (n=15). Nonsense errors, homonyms and spelling errors were present in 10.9% (n=14), 4.7% (n=6), and 0.8% (n=1) of notes, respectively. There were no suffix or dictionary errors. Inter-annotator agreement was 97.8%. CONCLUSIONS: This is the first estimate at classifying speech recognition errors in dictated emergency department notes. Speech recognition errors occur commonly with annunciation errors being the most frequent. Error rates were comparable if not lower than previous studies. 15% of errors were deemed critical, potentially leading to miscommunication that could affect patient care.
BACKGROUND: Physician use of computerized speech recognition (SR) technology has risen in recent years due to its ease of use and efficiency at the point of care. However, error rates between 10 and 23% have been observed, raising concern about the number of errors being entered into the permanent medical record, their impact on quality of care and medical liability that may arise. Our aim was to determine the incidence and types of SR errors introduced by this technology in the emergency department (ED). SETTING: Level 1 emergency department with 42,000 visits/year in a tertiary academic teaching hospital. METHODS: A random sample of 100 notes dictated by attending emergency physicians (EPs) using SR software was collected from the ED electronic health record between January and June 2012. Two board-certified EPs annotated the notes and conducted error analysis independently. An existing classification schema was adopted to classify errors into eight errors types. Critical errors deemed to potentially impact patient care were identified. RESULTS: There were 128 errors in total or 1.3 errors per note, and 14.8% (n=19) errors were judged to be critical. 71% of notes contained errors, and 15% contained one or more critical errors. Annunciation errors were the highest at 53.9% (n=69), followed by deletions at 18.0% (n=23) and added words at 11.7% (n=15). Nonsense errors, homonyms and spelling errors were present in 10.9% (n=14), 4.7% (n=6), and 0.8% (n=1) of notes, respectively. There were no suffix or dictionary errors. Inter-annotator agreement was 97.8%. CONCLUSIONS: This is the first estimate at classifying speech recognition errors in dictated emergency department notes. Speech recognition errors occur commonly with annunciation errors being the most frequent. Error rates were comparable if not lower than previous studies. 15% of errors were deemed critical, potentially leading to miscommunication that could affect patient care.
Authors: Atif Zafar; Burke Mamlin; Susan Perkins; Anne M Belsito; J Marc Overhage; Clement J McDonald Journal: Int J Med Inform Date: 2004-09 Impact factor: 4.046
Authors: Stephanie E Pollard; Pamela M Neri; Allison R Wilcox; Lynn A Volk; Deborah H Williams; Gordon D Schiff; Harley Z Ramelson; David W Bates Journal: Int J Med Inform Date: 2012-04-28 Impact factor: 4.046
Authors: Steven J Davidson; Frank L Zwemer; Larry A Nathanson; Kenneth N Sable; Abu N G A Khan Journal: Acad Emerg Med Date: 2004-11 Impact factor: 3.451
Authors: Kevin J Lybarger; Mari Ostendorf; Eve Riskin; Thomas H Payne; Andrew A White; Meliha Yetisgen Journal: Appl Clin Inform Date: 2018-10-17 Impact factor: 2.342
Authors: Maxim Topaz; Adam Schaffer; Kenneth H Lai; Zfania Tom Korach; Jonathan Einbinder; Li Zhou Journal: J Med Syst Date: 2018-07-09 Impact factor: 4.460
Authors: Scott Monteith; Tasha Glenn; John Geddes; Peter C Whybrow; Eric Achtyes; Michael Bauer Journal: Curr Psychiatry Rep Date: 2022-10-10 Impact factor: 8.081
Authors: Li Zhou; Suzanne V Blackley; Leigh Kowalski; Raymond Doan; Warren W Acker; Adam B Landman; Evgeni Kontrient; David Mack; Marie Meteer; David W Bates; Foster R Goss Journal: JAMA Netw Open Date: 2018-07-06