Can Machine-learning Techniques Be Used for 5-year Survival Prediction of Patients With Chondrosarcoma?

BACKGROUND: Several studies have identified prognostic factors for patients with chondrosarcoma, but there are few studies investigating the accuracy of computationally intensive methods such as machine learning. Machine learning is a type of artificial intelligence that enables computers to learn from data. Studies using machine learning are potentially appealing, because of its possibility to explore complex patterns in data and to improve its models over time. QUESTIONS/PURPOSES: The purposes of this study were (1) to develop machine-learning algorithms for the prediction of 5-year survival in patients with chondrosarcoma; and (2) to deploy the best algorithm as an accessible web-based app for clinical use.
METHODS: All patients with a microscopically confirmed diagnosis of conventional or dedifferentiated chondrosarcoma were extracted from the Surveillance, Epidemiology, and End Results (SEER) Registry from 2000 to 2010. SEER covers approximately 30% of the US population and consists of demographic, tumor characteristic, treatment, and outcome data. In total, 1554 patients met the inclusion criteria. Mean age at diagnosis was 52 years (SD 17), ranging from 7 to 102 years; 813 of the 1554 patients were men (55%); and mean tumor size was 8 cm (SD 6), ranging from 0.1 cm to 50 cm. Exact size was missing in 340 of 1544 patients (22%), grade in 88 of 1544 (6%), tumor extension in 41 of 1544 (3%), and race in 16 of 1544 (1%). Data for 1-, 3-, 5-, and 10-year overall survival were available for 1533 (99%), 1512 (98%), 1487 (96%), and 977 (63%) patients, respectively. One-year survival was 92%, 3-year survival was 82%, 5-year survival was 76%, and 10-year survival was 54%. Missing data were imputed using the nonparametric missForest method. Boosted decision tree, support vector machine, Bayes point machine, and neural network models were developed for 5-year survival. These models were chosen as a result of their capability of predicting two outcomes based on prior work on machine-learning models for binary classification. The models were assessed by discrimination, calibration, and overall performance. The c-statistic is a measure of discrimination. It ranges from 0.5 to 1.0 with 1.0 being perfect discrimination and 0.5 that the model is no better than chance at making a prediction. The Brier score measures the squared difference between the predicted probability and the actual outcome. A Brier score of 0 indicates perfect prediction, whereas a Brier score of 1 indicates the poorest prediction. The Brier scores of the models are compared with the null model, which is calculated by assigning each patient a probability equal to the prevalence of the outcome.
RESULTS: Four models for 5-year survival were developed with c-statistics ranging from 0.846 to 0.868 and Brier scores ranging from 0.117 to 0.135 with a null model Brier score of 0.182. The Bayes point machine was incorporated into a freely available web-based application. This application can be accessed through https://sorg-apps.shinyapps.io/chondrosarcoma/.
CONCLUSIONS: Although caution is warranted, because the prediction model has not been validated yet, healthcare providers could use the online prediction tool in daily practice when survival prediction of patients with chondrosarcoma is desired. Future studies should seek to validate the developed prediction model. LEVEL OF EVIDENCE: Level III, prognostic study.