When is a diagnosis worth making? A statistical comparison of two prediction strategies.

For a setting in which the sole goal is to predict a criterion accurately, two strategies are compared, (1) multiple linear regression directly from a set of predictors and (2) using predictors to diagnose individuals and then using only the diagnosis to predict the criterion. These are mathematical models of two common methods for making clinical predictions. This article derives equations for each statistical strategy's validity (and a formula comparing them, for a special case). Prediction accuracies are compared over a simplified but broad parameter space and four conclusions follow. Changing disorder prevalences (in the range 0.1 to 0.5) have small effect on the relative preferability of the two strategies; the effect of varying prevalence differs according to population separation on predictors and criterion. As within-population covariances of predictors with criterion decline below zero (assuming variables are scaled so that the less common population scores higher on predictors and criterion), diagnoses become increasingly preferable as a prediction strategy; as they rise above zero the opposite trend is observed. As populations become better separated on predictors or criterion, diagnoses compare more favorably. Most importantly, over almost all of the parameter space which one would expect to encounter in clinical psychology and psychiatry, multiple linear regression is much superior.