A simulation study of predictive ability measures in a survival model I: explained variation measures.

Measures of predictive ability play an important role in quantifying the clinical significance of prognostic factors. Several measures have been proposed to evaluate the predictive ability of survival models in the last two decades, but no single measure is consistently used. The proposed measures can be classified into the following categories: explained variation, explained randomness, and predictive accuracy. The three categories are conceptually different and are based on different principles. Several new measures have been proposed since Schemper and Stare's study in 1996 on some of the existing measures. This paper is the first of two papers that study the proposed measures systematically by applying a set of criteria that a measure of predictive ability should possess in the context of survival analysis. The present paper focuses on the explained variation category, and part II studies the proposed measures in the other categories. Simulation studies are used to examine the performance of five explained variation measures with respect to these criteria, discussing their strengths and shortcomings. Our simulation studies show that the measures proposed by Kent and O'Quigley, R(PM)(2), and Royston and Sauerbrei, R(D)(2), appear to be the best overall at quantifying predictive ability. However, it should be noted that neither measure is perfect; R(PM)(2) is sensitive to outliers and R(D)(2) to (marked) non-normality of the distribution of the prognostic index. The results show that the other measures perform poorly, primarily because they are adversely affected by censoring.