Christopher W Noel1,2,3, Rinku Sutradhar2,3, Lesley Gotlib Conn2,4, David Forner2,5, Wing C Chan3, Rui Fu2,3,4, Julie Hallet2,3,4,6, Natalie G Coburn2,3,4,6, Antoine Eskander1,2,3,4,6. 1. Department of Otolaryngology-Head and Neck Surgery, University of Toronto, Toronto, Ontario, Canada. 2. Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada. 3. ICES, Toronto, Ontario, Canada. 4. Evaluative Clinical Sciences, Sunnybrook Research Institute, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada. 5. Division of Otolaryngology-Head and Neck Surgery, Dalhousie University, Halifax, Nova Scotia, Canada. 6. Department of Surgical Oncology, Odette Cancer Centre, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.
Abstract
Importance: Patient-reported symptom burden was recently found to be associated with emergency department use and unplanned hospitalization (ED/Hosp) in patients with head and neck cancer. It was hypothesized that symptom scores could be combined with administrative health data to accurately risk stratify patients. Objective: To develop and validate a machine learning approach to predict future ED/Hosp in patients with head and neck cancer. Design, Setting, and Participants: This was a population-based predictive modeling study of patients in Ontario, Canada, diagnosed with head and neck cancer from January 2007 through March 2018. All outpatient clinical encounters were identified. Edmonton Symptom Assessment System (ESAS) scores and clinical and demographic factors were abstracted. Training and test cohorts were randomly generated in a 4:1 ratio. Various machine learning algorithms were explored, including (1) logistic regression using a least absolute shrinkage and selection operator, (2) random forest, (3) gradient boosting machine, (4) k-nearest neighbors, and (5) an artificial neural network. Data analysis was performed from September 2021 to January 2022. Main Outcomes and Measures: The main outcome was any 14-day ED/Hosp event following symptom assessment. The performance of each model was assessed on the test cohort using the area under the receiver operator characteristic (AUROC) curve and calibration plots. Shapley values were used to identify the variables with greatest contribution to the model. Results: The training cohort consisted of 9409 patients (mean [SD] age, 63.3 [10.9] years) undergoing 59 089 symptom assessments (80%). The remaining 2352 patients (mean [SD] age, 63.3 [11] years) and 14 193 symptom assessments were set aside as the test cohort (20%). Several models had high predictive accuracy, particularly the gradient boosting machine (validation AUROC, 0.80 [95% CI, 0.78-0.81]). A Youden-based cutoff corresponded to a validation sensitivity of 0.77 and specificity of 0.66. Patient-reported symptom scores were consistently identified as being the most predictive features within models. A second model built only with symptom severity data had an AUROC of 0.72 (95% CI, 0.70-0.74). Conclusions and Relevance: In this study, machine learning approaches predicted with a high degree of accuracy ED/Hosp in patients with head and neck cancer. These tools could be used to accurately risk stratify patients and may help direct targeted intervention.
Importance: Patient-reported symptom burden was recently found to be associated with emergency department use and unplanned hospitalization (ED/Hosp) in patients with head and neck cancer. It was hypothesized that symptom scores could be combined with administrative health data to accurately risk stratify patients. Objective: To develop and validate a machine learning approach to predict future ED/Hosp in patients with head and neck cancer. Design, Setting, and Participants: This was a population-based predictive modeling study of patients in Ontario, Canada, diagnosed with head and neck cancer from January 2007 through March 2018. All outpatient clinical encounters were identified. Edmonton Symptom Assessment System (ESAS) scores and clinical and demographic factors were abstracted. Training and test cohorts were randomly generated in a 4:1 ratio. Various machine learning algorithms were explored, including (1) logistic regression using a least absolute shrinkage and selection operator, (2) random forest, (3) gradient boosting machine, (4) k-nearest neighbors, and (5) an artificial neural network. Data analysis was performed from September 2021 to January 2022. Main Outcomes and Measures: The main outcome was any 14-day ED/Hosp event following symptom assessment. The performance of each model was assessed on the test cohort using the area under the receiver operator characteristic (AUROC) curve and calibration plots. Shapley values were used to identify the variables with greatest contribution to the model. Results: The training cohort consisted of 9409 patients (mean [SD] age, 63.3 [10.9] years) undergoing 59 089 symptom assessments (80%). The remaining 2352 patients (mean [SD] age, 63.3 [11] years) and 14 193 symptom assessments were set aside as the test cohort (20%). Several models had high predictive accuracy, particularly the gradient boosting machine (validation AUROC, 0.80 [95% CI, 0.78-0.81]). A Youden-based cutoff corresponded to a validation sensitivity of 0.77 and specificity of 0.66. Patient-reported symptom scores were consistently identified as being the most predictive features within models. A second model built only with symptom severity data had an AUROC of 0.72 (95% CI, 0.70-0.74). Conclusions and Relevance: In this study, machine learning approaches predicted with a high degree of accuracy ED/Hosp in patients with head and neck cancer. These tools could be used to accurately risk stratify patients and may help direct targeted intervention.
Authors: Ethan Basch; Allison M Deal; Amylou C Dueck; Howard I Scher; Mark G Kris; Clifford Hudis; Deborah Schrag Journal: JAMA Date: 2017-07-11 Impact factor: 56.272
Authors: A Eskander; M K Krzyzanowska; H D Fischer; N Liu; P C Austin; J C Irish; D J Enepekides; J Lee; E Gutierrez; E Lockhart; M Raphael; S Singh Journal: Oral Oncol Date: 2018-06-19 Impact factor: 5.337
Authors: Ethan Basch; Allison M Deal; Mark G Kris; Howard I Scher; Clifford A Hudis; Paul Sabbatini; Lauren Rogak; Antonia V Bennett; Amylou C Dueck; Thomas M Atkinson; Joanne F Chou; Dorothy Dulko; Laura Sit; Allison Barz; Paul Novotny; Michael Fruscione; Jeff A Sloan; Deborah Schrag Journal: J Clin Oncol Date: 2015-12-07 Impact factor: 44.544
Authors: Julienne E Bower; Kate Bak; Ann Berger; William Breitbart; Carmelita P Escalante; Patricia A Ganz; Hester Hill Schnipper; Christina Lacchetti; Jennifer A Ligibel; Gary H Lyman; Mohammed S Ogaily; William F Pirl; Paul B Jacobsen Journal: J Clin Oncol Date: 2014-04-14 Impact factor: 44.544