M Schumacher1, E Graf, T Gerds. 1. Institute of Medical Biometry and Medical Informatics, University Hospital Freiburg, Freiburg, Germany. ms@imbi.uni-freiburg.de
Abstract
OBJECTIVES: A lack of generally applicable tools for the assessment of predictions for survival data has to be recognized. Prediction error curves based on the Brier score that have been suggested as a sensible approach are illustrated by means of a case study. METHODS: The concept of predictions made in terms of conditional survival probabilities given the patient's covariates is introduced. Such predictions are derived from various statistical models for survival data including artificial neural networks. The idea of how the prediction error of a prognostic classification scheme can be followed over time is illustrated with the data of two studies on the prognosis of node positive breast cancer patients, one of them serving as an independent test data set. RESULTS AND CONCLUSIONS: The Brier score as a function of time is shown to be a valuable tool for assessing the predictive performance of prognostic classification schemes for survival data incorporating censored observations. Comparison with the prediction based on the pooled Kaplan Meier estimator yields a benchmark value for any classification scheme incorporating patient's covariate measurements. The problem of an overoptimistic assessment of prediction error caused by data-driven modelling as it is, for example, done with artificial neural nets can be circumvented by an assessment in an independent test data set.
OBJECTIVES: A lack of generally applicable tools for the assessment of predictions for survival data has to be recognized. Prediction error curves based on the Brier score that have been suggested as a sensible approach are illustrated by means of a case study. METHODS: The concept of predictions made in terms of conditional survival probabilities given the patient's covariates is introduced. Such predictions are derived from various statistical models for survival data including artificial neural networks. The idea of how the prediction error of a prognostic classification scheme can be followed over time is illustrated with the data of two studies on the prognosis of node positive breast cancerpatients, one of them serving as an independent test data set. RESULTS AND CONCLUSIONS: The Brier score as a function of time is shown to be a valuable tool for assessing the predictive performance of prognostic classification schemes for survival data incorporating censored observations. Comparison with the prediction based on the pooled Kaplan Meier estimator yields a benchmark value for any classification scheme incorporating patient's covariate measurements. The problem of an overoptimistic assessment of prediction error caused by data-driven modelling as it is, for example, done with artificial neural nets can be circumvented by an assessment in an independent test data set.
Authors: Peter G Hawkins; Yilun Sun; Robert T Dess; William C Jackson; Grace Sun; Nan Bi; Muneesh Tewari; James A Hayman; Gregory P Kalemkerian; Shirish M Gadgeel; Theodore S Lawrence; Randall K Ten Haken; Martha M Matuszak; Feng-Ming Spring Kong; Matthew J Schipper; Shruti Jolly Journal: J Cancer Res Clin Oncol Date: 2019-03-28 Impact factor: 4.553
Authors: Ewout W Steyerberg; Andrew J Vickers; Nancy R Cook; Thomas Gerds; Mithat Gonen; Nancy Obuchowski; Michael J Pencina; Michael W Kattan Journal: Epidemiology Date: 2010-01 Impact factor: 4.822
Authors: I Galea; C Lederer; A Neuhaus; P A Muraro; A Scalfari; N Koch-Henriksen; C Heesen; S Koepke; P Stellmann; H Albrecht; A Winkelmann; F Weber; E Bahn; M Hauser; G Edan; G Ebers; M Daumer Journal: Eur J Neurol Date: 2012-12-07 Impact factor: 6.089