Evaluating a new marker for risk prediction: decision analysis to the rescue.

In many areas of medicine risk prediction models are used to identify high-risk persons to receive treatment, with the goal of maximizing the ratio of benefits to harms. Thus there is considerable interest in evaluating markers to improve risk prediction. Many measures to evaluate a new marker for risk prediction are based solely on predictive accuracy including the odds ratio, change in the area under the receiver operating characteristic curve, and net reclassification improvement. However, predictive accuracy measures do not capture important clinical implications. Decision analysis comes to the rescue by including the ratio of the anticipated harm ("cost") of a false positive to the anticipated benefit of a true positive, which is transformed into a risk threshold (T) of indifference between treatment and no treatment. A decision-analytic measure of the "value" of a new marker is the number needed to test at a particular risk threshold, denoted NNTest(T), the minimum number of marker tests per true positive needed for risk prediction to be worthwhile. If NNTest(T) is acceptable given the invasiveness and adverse consequences of the test for the new marker, the new marker is recommended for inclusion in risk prediction. We provide a simple review of the derivation and computation of NNTest(T) from risk stratification tables and compare the minimum of NNTest(T), over risk thresholds, with measures of predictive accuracy in six studies. The results illustrate the advantages of this decision-analytic approach for evaluating a new marker for risk prediction.