Bias in discrepant analysis: when two wrongs don't make a right.

Imperfect reference standards (alloyed standards) often hinder evaluation of new diagnostic tests. Discrepant analysis, a two-stage modification of the standard evaluation of diagnostic tests, has been used to circumvent this problem. In discrepant analysis, additional testing is performed to resolve discrepant results of the new diagnostic test and the alloyed standard. This article demonstrates algebraically that sensitivity and specificity will be overestimated by discrepant analysis, even when a perfect gold standard is used to resolve the discrepant results. The magnitude of the bias depends on the true sensitivity and specificity of the new test and initial alloyed standard, the prevalence of disease in the study population, and the proportion of concordant errors between the two tests. The article also demonstrates substantial bias associated with the use of alloyed standard tests in both stages of discrepant analysis. This procedure should not be used routinely for evaluation of diagnostic tests.