Michael R Mathis1,2,3, Milo C Engoren1, Hyeon Joo1, Michael D Maile1, Keith D Aaronson4, Michael L Burns1,3, Michael W Sjoding2,3,5, Nicholas J Douville1, Allison M Janda1, Yaokun Hu1, Kayvan Najarian2,3, Sachin Kheterpal1,3. 1. From the Department of Anesthesiology. 2. Center for Computational Medicine and Bioinformatics, University of Michigan Health System, Ann Arbor, Michigan. 3. Michigan Integrated Center for Health Analytics and Medical Prediction, Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, Michigan. 4. Department of Internal Medicine - Cardiovascular Medicine Division, University of Michigan Health System, Ann Arbor, Michigan. 5. Department of Internal Medicine - Pulmonary and Critical Care Division, University of Michigan Health System, Ann Arbor, Michigan.
Abstract
BACKGROUND: Heart failure with reduced ejection fraction (HFrEF) is a condition imposing significant health care burden. Given its syndromic nature and often insidious onset, the diagnosis may not be made until clinical manifestations prompt further evaluation. Detecting HFrEF in precursor stages could allow for early initiation of treatments to modify disease progression. Granular data collected during the perioperative period may represent an underutilized method for improving the diagnosis of HFrEF. We hypothesized that patients ultimately diagnosed with HFrEF following surgery can be identified via machine-learning approaches using pre- and intraoperative data. METHODS: Perioperative data were reviewed from adult patients undergoing general anesthesia for major surgical procedures at an academic quaternary care center between 2010 and 2016. Patients with known HFrEF, heart failure with preserved ejection fraction, preoperative critical illness, or undergoing cardiac, cardiology, or electrophysiologic procedures were excluded. Patients were classified as healthy controls or undiagnosed HFrEF. Undiagnosed HFrEF was defined as lacking a HFrEF diagnosis preoperatively but establishing a diagnosis within 730 days postoperatively. Undiagnosed HFrEF patients were adjudicated by expert clinician review, excluding cases for which HFrEF was secondary to a perioperative triggering event, or any event not associated with HFrEF natural disease progression. Machine-learning models, including L1 regularized logistic regression, random forest, and extreme gradient boosting were developed to detect undiagnosed HFrEF, using perioperative data including 628 preoperative and 1195 intraoperative features. Training/validation and test datasets were used with parameter tuning. Test set model performance was evaluated using area under the receiver operating characteristic curve (AUROC), positive predictive value, and other standard metrics. RESULTS: Among 67,697 cases analyzed, 279 (0.41%) patients had undiagnosed HFrEF. The AUROC for the logistic regression model was 0.869 (95% confidence interval, 0.829-0.911), 0.872 (0.836-0.909) for the random forest model, and 0.873 (0.833-0.913) for the extreme gradient boosting model. The corresponding positive predictive values were 1.69% (1.06%-2.32%), 1.42% (0.85%-1.98%), and 1.78% (1.15%-2.40%), respectively. CONCLUSIONS: Machine-learning models leveraging perioperative data can detect undiagnosed HFrEF with good performance. However, the low prevalence of the disease results in a low positive predictive value, and for clinically meaningful sensitivity thresholds to be actionable, confirmatory testing with high specificity (eg, echocardiography or cardiac biomarkers) would be required following model detection. Future studies are necessary to externally validate algorithm performance at additional centers and explore the feasibility of embedding algorithms into the perioperative electronic health record for clinician use in real time.
BACKGROUND:Heart failure with reduced ejection fraction (HFrEF) is a condition imposing significant health care burden. Given its syndromic nature and often insidious onset, the diagnosis may not be made until clinical manifestations prompt further evaluation. Detecting HFrEF in precursor stages could allow for early initiation of treatments to modify disease progression. Granular data collected during the perioperative period may represent an underutilized method for improving the diagnosis of HFrEF. We hypothesized that patients ultimately diagnosed with HFrEF following surgery can be identified via machine-learning approaches using pre- and intraoperative data. METHODS: Perioperative data were reviewed from adult patients undergoing general anesthesia for major surgical procedures at an academic quaternary care center between 2010 and 2016. Patients with known HFrEF, heart failure with preserved ejection fraction, preoperative critical illness, or undergoing cardiac, cardiology, or electrophysiologic procedures were excluded. Patients were classified as healthy controls or undiagnosed HFrEF. Undiagnosed HFrEF was defined as lacking a HFrEF diagnosis preoperatively but establishing a diagnosis within 730 days postoperatively. Undiagnosed HFrEFpatients were adjudicated by expert clinician review, excluding cases for which HFrEF was secondary to a perioperative triggering event, or any event not associated with HFrEF natural disease progression. Machine-learning models, including L1 regularized logistic regression, random forest, and extreme gradient boosting were developed to detect undiagnosed HFrEF, using perioperative data including 628 preoperative and 1195 intraoperative features. Training/validation and test datasets were used with parameter tuning. Test set model performance was evaluated using area under the receiver operating characteristic curve (AUROC), positive predictive value, and other standard metrics. RESULTS: Among 67,697 cases analyzed, 279 (0.41%) patients had undiagnosed HFrEF. The AUROC for the logistic regression model was 0.869 (95% confidence interval, 0.829-0.911), 0.872 (0.836-0.909) for the random forest model, and 0.873 (0.833-0.913) for the extreme gradient boosting model. The corresponding positive predictive values were 1.69% (1.06%-2.32%), 1.42% (0.85%-1.98%), and 1.78% (1.15%-2.40%), respectively. CONCLUSIONS: Machine-learning models leveraging perioperative data can detect undiagnosed HFrEF with good performance. However, the low prevalence of the disease results in a low positive predictive value, and for clinically meaningful sensitivity thresholds to be actionable, confirmatory testing with high specificity (eg, echocardiography or cardiac biomarkers) would be required following model detection. Future studies are necessary to externally validate algorithm performance at additional centers and explore the feasibility of embedding algorithms into the perioperative electronic health record for clinician use in real time.
Authors: T H Lee; E R Marcantonio; C M Mangione; E J Thomas; C A Polanczyk; E F Cook; D J Sugarbaker; M C Donaldson; R Poss; K K Ho; L E Ludwig; A Pedan; L Goldman Journal: Circulation Date: 1999-09-07 Impact factor: 29.690
Authors: Michael F Aziz; Ansgar M Brambrink; David W Healy; Amy Wen Willett; Amy Shanks; Tyler Tremper; Leslie Jameson; Jacqueline Ragheb; Daniel A Biggs; William C Paganelli; Janavi Rao; Jerry L Epps; Douglas A Colquhoun; Patrick Bakke; Sachin Kheterpal Journal: Anesthesiology Date: 2016-10 Impact factor: 7.892
Authors: S Patrick Bender; William C Paganelli; Lyle P Gerety; William G Tharp; Amy M Shanks; Michelle Housey; Randal S Blank; Douglas A Colquhoun; Ana Fernandez-Bustamante; Leslie C Jameson; Sachin Kheterpal Journal: Anesth Analg Date: 2015-11 Impact factor: 5.108
Authors: Saul Blecker; Stuart D Katz; Leora I Horwitz; Gilad Kuperman; Hannah Park; Alex Gold; David Sontag Journal: JAMA Cardiol Date: 2016-12-01 Impact factor: 14.676
Authors: Timothy J W Dawes; Antonio de Marvao; Wenzhe Shi; Tristan Fletcher; Geoffrey M J Watson; John Wharton; Christopher J Rhodes; Luke S G E Howard; J Simon R Gibbs; Daniel Rueckert; Stuart A Cook; Martin R Wilkins; Declan P O'Regan Journal: Radiology Date: 2017-01-16 Impact factor: 11.105
Authors: Michael R Mathis; Robert B Schonberger; Elizabeth L Whitlock; Keith M Vogt; John E Lagorio; Keith A Jones; Joanne M Conroy; Sachin Kheterpal Journal: Anesth Analg Date: 2022-02-01 Impact factor: 6.627
Authors: Kevin G Pollock; Sara Sekelj; Ellie Johnston; Belinda Sandler; Nathan R Hill; Fu Siong Ng; Sadia Khan; Ayman Nassar; Usman Farooqui Journal: Int J Cardiol Heart Vasc Date: 2020-11-29