John Adeoye1, Liuling Hui2, Mohamad Koohi-Moghadam3, Jia Yan Tan4, Siu-Wai Choi4, Peter Thomson5. 1. Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China; Oral Cancer Research Group, Faculty of Dentistry, University of Hong Kong, Hong Kong, China. Electronic address: jaadeoye@hku.hk. 2. Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China. 3. Applied Oral Sciences and Community Dental Care, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China. 4. Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China; Oral Cancer Research Group, Faculty of Dentistry, University of Hong Kong, Hong Kong, China. 5. Oral and Maxillofacial Surgery, Faculty of Dentistry, The University of Hong Kong, Hong Kong, China; Oral Cancer Research Group, Faculty of Dentistry, University of Hong Kong, Hong Kong, China; College of Medicine and Dentistry, James Cook University, Queensland, Australia. Electronic address: peter.thomson1@jcu.edu.au.
Abstract
BACKGROUND: Applying machine learning to predicting oral cavity cancer prognosis is important in selecting candidates for aggressive treatment following diagnosis. However, models proposed so far have only considered cancer survival as discrete rather than dynamic outcomes. OBJECTIVES: To compare the model performance of different machine learning-based algorithms that incorporate time-to-event data. These algorithms included DeepSurv, DeepHit, neural net-extended time-dependent cox model (Cox-Time), and random survival forest (RSF). MATERIALS AND METHODS: Retrospective cohort of 313 oral cavity cancer patients were obtained from electronic health records. Models were trained on patient data following preprocessing. Predictors were based on demographic, clinicopathologic, and treatment information of the cases. Outcomes were the disease-specific and overall survival. Multivariable analyses were conducted to select significant prognostic features associated with tumor prognosis. Two models were generated per algorithm based on all-prognostic features and significant-prognostic features following statistical analysis. Concordance index (c-index) and integrated Brier scores were used as performance evaluators and model stability was assessed using intraclass correlation coefficients (ICC) calculated from these measures obtained from the cross-validation folds. RESULTS: While all models were satisfactory, better discriminatory performance and calibration was observed for disease-specific than overall survival (mean c-index: 0.85 vs 0.74; mean integrated Brier score: 0.12 vs 0.17). DeepSurv performed best in terms of discrimination for both outcomes (c-indices: 0.76 -0.89) while RSF produced better calibrated survival estimates (integrated Brier score: 0.06 -0.09). Model stability of the algorithms varied with the outcomes as Cox-Time had the best intraclass correlation coefficient (mean ICC: 1.00) for disease-specific survival while DeepSurv was most stable for overall survival prediction (mean ICC: 0.99). CONCLUSIONS: Machine learning algorithms based on time-to-event outcomes are successful in predicting oral cavity cancer prognosis with DeepSurv and RSF producing the best discriminative performance and calibration.
BACKGROUND: Applying machine learning to predicting oral cavity cancer prognosis is important in selecting candidates for aggressive treatment following diagnosis. However, models proposed so far have only considered cancer survival as discrete rather than dynamic outcomes. OBJECTIVES: To compare the model performance of different machine learning-based algorithms that incorporate time-to-event data. These algorithms included DeepSurv, DeepHit, neural net-extended time-dependent cox model (Cox-Time), and random survival forest (RSF). MATERIALS AND METHODS: Retrospective cohort of 313 oral cavity cancer patients were obtained from electronic health records. Models were trained on patient data following preprocessing. Predictors were based on demographic, clinicopathologic, and treatment information of the cases. Outcomes were the disease-specific and overall survival. Multivariable analyses were conducted to select significant prognostic features associated with tumor prognosis. Two models were generated per algorithm based on all-prognostic features and significant-prognostic features following statistical analysis. Concordance index (c-index) and integrated Brier scores were used as performance evaluators and model stability was assessed using intraclass correlation coefficients (ICC) calculated from these measures obtained from the cross-validation folds. RESULTS: While all models were satisfactory, better discriminatory performance and calibration was observed for disease-specific than overall survival (mean c-index: 0.85 vs 0.74; mean integrated Brier score: 0.12 vs 0.17). DeepSurv performed best in terms of discrimination for both outcomes (c-indices: 0.76 -0.89) while RSF produced better calibrated survival estimates (integrated Brier score: 0.06 -0.09). Model stability of the algorithms varied with the outcomes as Cox-Time had the best intraclass correlation coefficient (mean ICC: 1.00) for disease-specific survival while DeepSurv was most stable for overall survival prediction (mean ICC: 0.99). CONCLUSIONS: Machine learning algorithms based on time-to-event outcomes are successful in predicting oral cavity cancer prognosis with DeepSurv and RSF producing the best discriminative performance and calibration.