Shounak Datta1, Tyler J Loftus2, Matthew M Ruppert1, Chris Giordano3, Gilbert R Upchurch4, Parisa Rashidi5, Tezcan Ozrazgat-Baslanti1, Azra Bihorac6. 1. Department of Medicine, University of Florida, Gainesville, Florida; Precision and Intelligent Systems in Medicine (Prisma(P)), University of Florida, Gainesville, Florida. 2. Precision and Intelligent Systems in Medicine (Prisma(P)), University of Florida, Gainesville, Florida; Department of Surgery, University of Florida, Gainesville, Florida. 3. Department of Anesthesiology, University of Florida, Gainesville, Florida. 4. Department of Surgery, University of Florida, Gainesville, Florida. 5. Precision and Intelligent Systems in Medicine (Prisma(P)), University of Florida, Gainesville, Florida; Department of Biomedical Engineering, University of Florida, Gainesville, Florida. 6. Department of Medicine, University of Florida, Gainesville, Florida; Precision and Intelligent Systems in Medicine (Prisma(P)), University of Florida, Gainesville, Florida. Electronic address: abihorac@ufl.edu.
Abstract
BACKGROUND: Models that predict postoperative complications often ignore important intraoperative events and physiological changes. This study tested the hypothesis that accuracy, discrimination, and precision in predicting postoperative complications would improve when using both preoperative and intraoperative data input data compared with preoperative data alone. METHODS: This retrospective cohort analysis included 43,943 adults undergoing 52,529 inpatient surgeries at a single institution during a 5-y period. Random forest machine learning models in the validated MySurgeryRisk platform made patient-level predictions for seven postoperative complications and mortality occurring during hospital admission using electronic health record data and patient neighborhood characteristics. For each outcome, one model trained with preoperative data alone; one model trained with both preoperative and intraoperative data. Models were compared by accuracy, discrimination (expressed as area under the receiver operating characteristic curve), precision (expressed as area under the precision-recall curve), and reclassification indices. RESULTS: Machine learning models incorporating both preoperative and intraoperative data had greater accuracy, discrimination, and precision than models using preoperative data alone for predicting all seven postoperative complications (intensive care unit length of stay >48 h, mechanical ventilation >48 h, neurologic complications including delirium, cardiovascular complications, acute kidney injury, venous thromboembolism, and wound complications), and in-hospital mortality (accuracy: 88% versus 77%; area under the receiver operating characteristic curve: 0.93 versus 0.87; area under the precision-recall curve: 0.21 versus 0.15). Overall reclassification improvement was 2.4%-10.0% for complications and 11.2% for in-hospital mortality. CONCLUSIONS: Incorporating both preoperative and intraoperative data significantly increased the accuracy, discrimination, and precision of machine learning models predicting postoperative complications and mortality.
BACKGROUND: Models that predict postoperative complications often ignore important intraoperative events and physiological changes. This study tested the hypothesis that accuracy, discrimination, and precision in predicting postoperative complications would improve when using both preoperative and intraoperative data input data compared with preoperative data alone. METHODS: This retrospective cohort analysis included 43,943 adults undergoing 52,529 inpatient surgeries at a single institution during a 5-y period. Random forest machine learning models in the validated MySurgeryRisk platform made patient-level predictions for seven postoperative complications and mortality occurring during hospital admission using electronic health record data and patient neighborhood characteristics. For each outcome, one model trained with preoperative data alone; one model trained with both preoperative and intraoperative data. Models were compared by accuracy, discrimination (expressed as area under the receiver operating characteristic curve), precision (expressed as area under the precision-recall curve), and reclassification indices. RESULTS: Machine learning models incorporating both preoperative and intraoperative data had greater accuracy, discrimination, and precision than models using preoperative data alone for predicting all seven postoperative complications (intensive care unit length of stay >48 h, mechanical ventilation >48 h, neurologic complications including delirium, cardiovascular complications, acute kidney injury, venous thromboembolism, and wound complications), and in-hospital mortality (accuracy: 88% versus 77%; area under the receiver operating characteristic curve: 0.93 versus 0.87; area under the precision-recall curve: 0.21 versus 0.15). Overall reclassification improvement was 2.4%-10.0% for complications and 11.2% for in-hospital mortality. CONCLUSIONS: Incorporating both preoperative and intraoperative data significantly increased the accuracy, discrimination, and precision of machine learning models predicting postoperative complications and mortality.
Authors: Brian J Douthit; Rachel L Walden; Kenrick Cato; Cynthia P Coviak; Christopher Cruz; Fabio D'Agostino; Thompson Forbes; Grace Gao; Theresa A Kapetanovic; Mikyoung A Lee; Lisiane Pruinelli; Mary A Schultz; Ann Wieben; Alvin D Jeffery Journal: Appl Clin Inform Date: 2022-02-09 Impact factor: 2.342
Authors: Xinyu Yan; Jeff Goldsmith; Sumit Mohan; Zachary A Turnbull; Robert E Freundlich; Frederic T Billings; Ravi P Kiran; Guohua Li; Minjae Kim Journal: Anesth Analg Date: 2022-01-01 Impact factor: 5.108
Authors: Danielle E Soranno; Azra Bihorac; Stuart L Goldstein; Kianoush B Kashani; Shina Menon; Girish N Nadkarni; Javier A Neyra; Neesh I Pannu; Karandeep Singh; Jorge Cerda; Jay L Koyner Journal: Kidney360 Date: 2021-11-18
Authors: Amanda C Filiberto; Tyler J Loftus; Craig T Elder; Sara Hensley; Amanda Frantz; Phillip Efron; Tezcan Ozrazgat-Baslanti; Azra Bihorac; Gilbert R Upchurch; Michol A Cooper Journal: Surgery Date: 2021-05-07 Impact factor: 4.348
Authors: Amanda C Filiberto; Tezcan Ozrazgat-Baslanti; Tyler J Loftus; Ying-Chih Peng; Shounak Datta; Philip Efron; Gilbert R Upchurch; Azra Bihorac; Michol A Cooper Journal: Surgery Date: 2021-02-27 Impact factor: 4.348
Authors: Tyler J Loftus; Jeremy A Balch; Matthew M Ruppert; Patrick J Tighe; William R Hogan; Parisa Rashidi; Gilbert R Upchurch; Azra Bihorac Journal: Ann Surg Date: 2022-02-01 Impact factor: 13.787