Benjamin T Miller1, Aldo Fafaj2, Luciano Tastaldi2, Hemasat Alkhatib2, Samuel Zolin2, Raha AlMarzooqi2, Chao Tu3, Diya Alaedeen4, Ajita S Prabhu2, David M Krpata2, Michael J Rosen2, Clayton C Petro2. 1. Cleveland Clinic Center for Abdominal Core Health, Department of General Surgery, Digestive Disease and Surgery Institute, The Cleveland Clinic Foundation, 9500 Euclid Avenue, A-100, Cleveland, OH, 44195, USA. millerb35@ccf.org. 2. Cleveland Clinic Center for Abdominal Core Health, Department of General Surgery, Digestive Disease and Surgery Institute, The Cleveland Clinic Foundation, 9500 Euclid Avenue, A-100, Cleveland, OH, 44195, USA. 3. Department of Quantitative Health Sciences, Lerner Research Institute, The Cleveland Clinic Foundation, 9500 Euclid Avenue, Cleveland, OH, 44195, USA. 4. Department of General Surgery, Digestive Disease and Surgery Institute, Fairview Hospital, The Cleveland Clinic Foundation, 18101 Lorain Avenue, Cleveland, OH, 44111, USA.
Abstract
INTRODUCTION: Collecting accurate operative details remains a limitation of surgical research. Surgeon-entered data in clinical registries offers one solution, but natural language processing (NLP) has emerged as a modality for automating manual chart review (MCR). This study aims to compare the accuracy and efficiency of NLP and MCR with a surgeon-entered, prospective registry data in determining the rate of gross bile spillage (GBS) during cholecystectomy. METHODS: Bile spillage rates were abstracted from an institutional, surgeon-entered clinical registry from July 2018 to January 2019. These rates were compared to those documented in the electronic medical record (EMR) using NLP and MCR to determine the sensitivity, specificity, and efficiency of each approach. RESULTS: Of the 782 registry entries, 191 cases (24.4%) had surgeon-reported bile spillage. MCR identified bile spillage in 121 cases (15.6%); however, bile spillage information was either missing or ambiguous in 454 cases (58.1%). NLP identified 99 cases (12.7%) of bile spillage. Data abstraction times for the registry, NLP, and MCR were 3 min, 5 min, and 12 h, respectively. When compared to the registry, MCR was 45% sensitive and 94% specific, while NLP was 27.2% sensitive and 92% specific for detecting bile spillage. These differences were significant (X2 = 19.446, P = < 0.001). CONCLUSION: Operative details, such as GBS, may not be abstracted by NLP or MCR if not clearly documented in the EMR. Clinical registries capture operative details, but they rely on surgeons to input the data.
INTRODUCTION: Collecting accurate operative details remains a limitation of surgical research. Surgeon-entered data in clinical registries offers one solution, but natural language processing (NLP) has emerged as a modality for automating manual chart review (MCR). This study aims to compare the accuracy and efficiency of NLP and MCR with a surgeon-entered, prospective registry data in determining the rate of gross bile spillage (GBS) during cholecystectomy. METHODS: Bile spillage rates were abstracted from an institutional, surgeon-entered clinical registry from July 2018 to January 2019. These rates were compared to those documented in the electronic medical record (EMR) using NLP and MCR to determine the sensitivity, specificity, and efficiency of each approach. RESULTS: Of the 782 registry entries, 191 cases (24.4%) had surgeon-reported bile spillage. MCR identified bile spillage in 121 cases (15.6%); however, bile spillage information was either missing or ambiguous in 454 cases (58.1%). NLP identified 99 cases (12.7%) of bile spillage. Data abstraction times for the registry, NLP, and MCR were 3 min, 5 min, and 12 h, respectively. When compared to the registry, MCR was 45% sensitive and 94% specific, while NLP was 27.2% sensitive and 92% specific for detecting bile spillage. These differences were significant (X2 = 19.446, P = < 0.001). CONCLUSION: Operative details, such as GBS, may not be abstracted by NLP or MCR if not clearly documented in the EMR. Clinical registries capture operative details, but they rely on surgeons to input the data.
Authors: Nastassia Tvardik; Ivan Kergourlay; André Bittar; Frédérique Segond; Stefan Darmoni; Marie-Hélène Metzger Journal: Int J Med Inform Date: 2018-06-06 Impact factor: 4.046
Authors: Thomas Peponis; Trine G Eskesen; Tomaz Mesar; Noelle Saillant; Haytham M A Kaafarani; D Dante Yeh; Peter J Fagenholz; Marc A de Moya; David R King; George C Velmahos Journal: J Am Coll Surg Date: 2018-03-02 Impact factor: 6.113
Authors: B K Poulose; S Roll; J W Murphy; B D Matthews; B Todd Heniford; G Voeller; W W Hope; M I Goldblatt; G L Adrales; M J Rosen Journal: Hernia Date: 2016-03-02 Impact factor: 4.739