BACKGROUND: To avoid complications associated with under- or overtreatment of patients with skeletal metastases, doctors need accurate survival estimates. Unfortunately, prognostic models for patients with skeletal metastases of the extremities are lacking, and physician-based estimates are generally inaccurate. QUESTIONS/PURPOSES: We developed three types of prognostic models and compared them using calibration plots, receiver operating characteristic (ROC) curves, and decision curve analysis to determine which one is best suited for clinical use. METHODS: A training set consisted of 189 patients who underwent surgery for skeletal metastases. We created models designed to predict 3- and 12-month survival using three methods: an Artificial Neural Network (ANN), a Bayesian Belief Network (BBN), and logistic regression. We then performed crossvalidation and compared the models in three ways: calibration plots plotting predicted against actual risk; area under the ROC curve (AUC) to discriminate the probability that a patient who died has a higher predicted probability of death compared to a patient who did not die; and decision curve analysis to quantify the clinical consequences of over- or undertreatment. RESULTS: All models appeared to be well calibrated, with the exception of the BBN, which underestimated 3-month survival at lower probability estimates. The ANN models had the highest discrimination, with an AUC of 0.89 and 0.93, respectively, for the 3- and 12-month models. Decision analysis revealed all models could be used clinically, but the ANN models consistently resulted in the highest net benefit, outperforming the BBN and logistic regression models. CONCLUSIONS: Our observations suggest use of the ANN model to aid decisions about surgery would lead to better patient outcomes than other alternative approaches to decision making. LEVEL OF EVIDENCE: Level II, prognostic study. See Instructions for Authors for a complete description of levels of evidence.
BACKGROUND: To avoid complications associated with under- or overtreatment of patients with skeletal metastases, doctors need accurate survival estimates. Unfortunately, prognostic models for patients with skeletal metastases of the extremities are lacking, and physician-based estimates are generally inaccurate. QUESTIONS/PURPOSES: We developed three types of prognostic models and compared them using calibration plots, receiver operating characteristic (ROC) curves, and decision curve analysis to determine which one is best suited for clinical use. METHODS: A training set consisted of 189 patients who underwent surgery for skeletal metastases. We created models designed to predict 3- and 12-month survival using three methods: an Artificial Neural Network (ANN), a Bayesian Belief Network (BBN), and logistic regression. We then performed crossvalidation and compared the models in three ways: calibration plots plotting predicted against actual risk; area under the ROC curve (AUC) to discriminate the probability that a patient who died has a higher predicted probability of death compared to a patient who did not die; and decision curve analysis to quantify the clinical consequences of over- or undertreatment. RESULTS: All models appeared to be well calibrated, with the exception of the BBN, which underestimated 3-month survival at lower probability estimates. The ANN models had the highest discrimination, with an AUC of 0.89 and 0.93, respectively, for the 3- and 12-month models. Decision analysis revealed all models could be used clinically, but the ANN models consistently resulted in the highest net benefit, outperforming the BBN and logistic regression models. CONCLUSIONS: Our observations suggest use of the ANN model to aid decisions about surgery would lead to better patient outcomes than other alternative approaches to decision making. LEVEL OF EVIDENCE: Level II, prognostic study. See Instructions for Authors for a complete description of levels of evidence.
Authors: K Jayasurya; G Fung; S Yu; C Dehing-Oberije; D De Ruysscher; A Hope; W De Neve; Y Lievens; P Lambin; A L A J Dekker Journal: Med Phys Date: 2010-04 Impact factor: 4.071
Authors: Saminathan S Nathan; John H Healey; Danilo Mellano; Bang Hoang; Isobel Lewis; Carol D Morris; Edward A Athanasian; Patrick J Boland Journal: J Clin Oncol Date: 2005-09-01 Impact factor: 44.544
Authors: J C Weeks; E F Cook; S J O'Day; L M Peterson; N Wenger; D Reding; F E Harrell; P Kussin; N V Dawson; A F Connors; J Lynn; R S Phillips Journal: JAMA Date: 1998-06-03 Impact factor: 56.272
Authors: Benjamin K Potter; Jonathan A Forsberg; Elizabeth Silvius; Matthew Wagner; Vivek Khatri; Seth A Schobel; Arnaud J Belard; Amy C Weintrob; David R Tribble; Eric A Elster Journal: Mil Med Date: 2019-01-01 Impact factor: 1.437
Authors: Quirina C B S Thio; Aditya V Karhade; Paul T Ogink; Kevin A Raskin; Karen De Amorim Bernstein; Santiago A Lozano Calderon; Joseph H Schwab Journal: Clin Orthop Relat Res Date: 2018-10 Impact factor: 4.176
Authors: Keith A Alfieri; Benjamin K Potter; Thomas A Davis; Matthew B Wagner; Eric A Elster; Jonathan A Forsberg Journal: Clin Orthop Relat Res Date: 2015-09 Impact factor: 4.176
Authors: Adam J Bevevino; Jonathan F Dickens; Benjamin K Potter; Theodora Dworak; Wade Gordon; Jonathan A Forsberg Journal: Clin Orthop Relat Res Date: 2014-10 Impact factor: 4.176
Authors: Ashley B Anderson; Rikard Wedin; Nicola Fabbri; Patrick Boland; John Healey; Jonathan A Forsberg Journal: Clin Orthop Relat Res Date: 2020-04 Impact factor: 4.755