BACKGROUND: The identification of surgical site infections for infection surveillance in hospitals depends on the manual abstraction of medical records and, for research purposes, depends mainly on the use of administrative or claims data. The objective of this study was to determine whether automating the abstraction process with natural language processing (NLP)-based models that analyze the free-text notes of the medical record can identify surgical site infections with predictive abilities that match the manual abstraction process and that surpass surgical site infection identification from administrative data. METHODS: We used surgical site infection surveillance data compiled by the infection prevention team to identify surgical site infections among patients undergoing orthopaedic surgical procedures at a tertiary care academic medical center from 2011 to 2017. We compiled a list of keywords suggestive of surgical site infections, and we used NLP to identify occurrences of these keywords and their grammatical variants in the free-text notes of the medical record. The key outcome was a binary indicator of whether a surgical site infection occurred. We estimated 7 incremental multivariable logistic regression models using a combination of administrative and NLP-derived variables. We split the analytic cohort into training (80%) and testing data sets (20%), and we used a tenfold cross-validation approach. The main analytic cohort included 172 surgical site infection cases and 200 controls that were repeatedly and randomly selected from a pool of 1,407 controls. RESULTS: For Model 1 (variables from administrative data only), the sensitivity was 68% and the positive predictive value was 70%; for Model 4 (with NLP 5-grams [distinct sequences of 5 contiguous words] from the medical record), the sensitivity was 97% and the positive predictive value was 97%; and for Model 7 (a combination of Models 1 and 4), the sensitivity was 97% and the positive predictive value was 97%. Thus, NLP-based models identified 97% of surgical site infections identified by manual abstraction with high precision and 43% more surgical site infections compared with models that used administrative data only. CONCLUSIONS: Models that used NLP keywords achieved predictive abilities that were comparable with the manual abstraction process and were superior to models that used administrative data only. NLP has the potential to automate and aid accurate surgical site infection identification and, thus, play an important role in their prevention. CLINICAL RELEVANCE: This study examines NLP's potential to automate the identification of surgical site infections. This automation can potentially aid the prevention and early identification of these surgical complications, thereby reducing their adverse clinical and economic impact.
BACKGROUND: The identification of surgical site infections for infection surveillance in hospitals depends on the manual abstraction of medical records and, for research purposes, depends mainly on the use of administrative or claims data. The objective of this study was to determine whether automating the abstraction process with natural language processing (NLP)-based models that analyze the free-text notes of the medical record can identify surgical site infections with predictive abilities that match the manual abstraction process and that surpass surgical site infection identification from administrative data. METHODS: We used surgical site infection surveillance data compiled by the infection prevention team to identify surgical site infections among patients undergoing orthopaedic surgical procedures at a tertiary care academic medical center from 2011 to 2017. We compiled a list of keywords suggestive of surgical site infections, and we used NLP to identify occurrences of these keywords and their grammatical variants in the free-text notes of the medical record. The key outcome was a binary indicator of whether a surgical site infection occurred. We estimated 7 incremental multivariable logistic regression models using a combination of administrative and NLP-derived variables. We split the analytic cohort into training (80%) and testing data sets (20%), and we used a tenfold cross-validation approach. The main analytic cohort included 172 surgical site infection cases and 200 controls that were repeatedly and randomly selected from a pool of 1,407 controls. RESULTS: For Model 1 (variables from administrative data only), the sensitivity was 68% and the positive predictive value was 70%; for Model 4 (with NLP 5-grams [distinct sequences of 5 contiguous words] from the medical record), the sensitivity was 97% and the positive predictive value was 97%; and for Model 7 (a combination of Models 1 and 4), the sensitivity was 97% and the positive predictive value was 97%. Thus, NLP-based models identified 97% of surgical site infections identified by manual abstraction with high precision and 43% more surgical site infections compared with models that used administrative data only. CONCLUSIONS: Models that used NLP keywords achieved predictive abilities that were comparable with the manual abstraction process and were superior to models that used administrative data only. NLP has the potential to automate and aid accurate surgical site infection identification and, thus, play an important role in their prevention. CLINICAL RELEVANCE: This study examines NLP's potential to automate the identification of surgical site infections. This automation can potentially aid the prevention and early identification of these surgical complications, thereby reducing their adverse clinical and economic impact.
Authors: Eyal Zimlichman; Daniel Henderson; Orly Tamir; Calvin Franz; Peter Song; Cyrus K Yamin; Carol Keohane; Charles R Denham; David W Bates Journal: JAMA Intern Med Date: 2013 Dec 9-23 Impact factor: 21.873
Authors: Ryan P Merkow; Mila H Ju; Jeanette W Chung; Bruce L Hall; Mark E Cohen; Mark V Williams; Thomas C Tsai; Clifford Y Ko; Karl Y Bilimoria Journal: JAMA Date: 2015-02-03 Impact factor: 56.272
Authors: Deverick J Anderson; Kelly Podgorny; Sandra I Berríos-Torres; Dale W Bratzler; E Patchen Dellinger; Linda Greene; Ann-Christine Nyquist; Lisa Saiman; Deborah S Yokoe; Lisa L Maragakis; Keith S Kaye Journal: Infect Control Hosp Epidemiol Date: 2014-06 Impact factor: 3.254
Authors: Maaike S M van Mourik; Annet Troelstra; Wouter W van Solinge; Karel G M Moons; Marc J M Bonten Journal: Clin Infect Dis Date: 2013-03-26 Impact factor: 9.079
Authors: Fern FitzHenry; Harvey J Murff; Michael E Matheny; Nancy Gentry; Elliot M Fielstein; Steven H Brown; Ruth M Reeves; Dominik Aronsky; Peter L Elkin; Vincent P Messina; Theodore Speroff Journal: Med Care Date: 2013-06 Impact factor: 2.983
Authors: Shelley S Magill; Jonathan R Edwards; Wendy Bamberg; Zintars G Beldavs; Ghinwa Dumyati; Marion A Kainer; Ruth Lynfield; Meghan Maloney; Laura McAllister-Hollod; Joelle Nadle; Susan M Ray; Deborah L Thompson; Lucy E Wilson; Scott K Fridkin Journal: N Engl J Med Date: 2014-03-27 Impact factor: 91.245
Authors: Thomas G Myers; Prem N Ramkumar; Benjamin F Ricciardi; Kenneth L Urish; Jens Kipper; Constantinos Ketonis Journal: J Bone Joint Surg Am Date: 2020-05-06 Impact factor: 5.284
Authors: Marie K Reumann; Benedikt J Braun; Maximilian M Menger; Fabian Springer; Johann Jazewitsch; Tobias Schwarz; Andreas Nüssler; Tina Histing; Mika F R Rollmann Journal: Unfallchirurgie (Heidelb) Date: 2022-07-09
Authors: Elham Sagheb; Taghi Ramazanian; Ahmad P Tafti; Sunyang Fu; Walter K Kremers; Daniel J Berry; David G Lewallen; Sunghwan Sohn; Hilal Maradit Kremers Journal: J Arthroplasty Date: 2020-10-10 Impact factor: 4.757
Authors: Olivier Q Groot; Michiel E R Bongers; Paul T Ogink; Joeky T Senders; Aditya V Karhade; Jos A M Bramer; Jorrit-Jan Verlaan; Joseph H Schwab Journal: Clin Orthop Relat Res Date: 2020-12 Impact factor: 4.755
Authors: Sunyang Fu; Cody C Wyles; Douglas R Osmon; Martha L Carvour; Elham Sagheb; Taghi Ramazanian; Walter K Kremers; David G Lewallen; Daniel J Berry; Sunghwan Sohn; Hilal Maradit Kremers Journal: J Arthroplasty Date: 2020-08-05 Impact factor: 4.757