Dennis L Shung1, Benjamin Au2, Richard Andrew Taylor2, J Kenneth Tay3, Stig B Laursen4, Adrian J Stanley5, Harry R Dalton6, Jeffrey Ngu7, Michael Schultz8, Loren Laine9. 1. Yale School of Medicine, New Haven, Connecticut. Electronic address: dennis.shung@yale.edu. 2. Yale School of Medicine, New Haven, Connecticut. 3. Stanford University, Palo Alto, California. 4. Odense University Hospital, Odense, Denmark. 5. Glasgow Royal Infirmary, Glasgow, United Kingdom. 6. Royal Cornwall Hospital, Cornwall, United Kingdom. 7. Christchurch Hospital, Christchurch, New Zealand. 8. Dunedin Hospital, Dunedin, New Zealand. 9. Yale School of Medicine, New Haven, Connecticut; Veterans Affairs Connecticut Healthcare System, West Haven, Connecticut. Electronic address: loren.laine@yale.edu.
Abstract
BACKGROUND & AIMS: Scoring systems are suboptimal for determining risk in patients with upper gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate the risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems. METHODS: We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (the United States, Scotland, England, and Denmark; n = 1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n = 399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis. RESULTS: The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for the GBS (P = .001), 0.73 for Rockall score (P < .001), and 0.78 for AIMS65 score (P < .001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P = .004), the Rockall score with an AUC of 0.66 (P < .001), and the AIMS65 with an AUC of 0.64 (P < .001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model versus 12% with GBS (P < .001). CONCLUSIONS: We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.
BACKGROUND & AIMS: Scoring systems are suboptimal for determining risk in patients with upper gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate the risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems. METHODS: We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (the United States, Scotland, England, and Denmark; n = 1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n = 399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis. RESULTS: The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for the GBS (P = .001), 0.73 for Rockall score (P < .001), and 0.78 for AIMS65 score (P < .001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P = .004), the Rockall score with an AUC of 0.66 (P < .001), and the AIMS65 with an AUC of 0.64 (P < .001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model versus 12% with GBS (P < .001). CONCLUSIONS: We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.
Authors: G A Paspatis; E Matrella; A Kapsoritakis; C Leontithis; N Papanikolaou; G J Chlouverakis; E Kouroumalis Journal: Eur J Gastroenterol Hepatol Date: 2000-11 Impact factor: 2.566
Authors: F Perry Wilson; Michael Shashaty; Jeffrey Testani; Iram Aqeel; Yuliya Borovskiy; Susan S Ellenberg; Harold I Feldman; Hilda Fernandez; Yevgeniy Gitelman; Jennie Lin; Dan Negoianu; Chirag R Parikh; Peter P Reese; Richard Urbani; Barry Fuchs Journal: Lancet Date: 2015-02-26 Impact factor: 79.321
Authors: P Czernichow; P Hochain; J B Nousbaum; J M Raymond; A Rudelli; J L Dupas; M Amouretti; H Gouérou; M H Capron; H Herman; R Colin Journal: Eur J Gastroenterol Hepatol Date: 2000-02 Impact factor: 2.566
Authors: John R Saltzman; Ying P Tabak; Brian H Hyett; Xiaowu Sun; Anne C Travis; Richard S Johannes Journal: Gastrointest Endosc Date: 2011-09-10 Impact factor: 9.427