Gains in accuracy from replicated readings of diagnostic images: prediction and assessment in terms of ROC analysis.

From the perspective of signal detection theory, human variation in image reading degrades diagnostic accuracy by broadening the statistical distributions of perceived evidence upon which decisions were based. A new multivariate "random-effects" model formulates the total variation in a diagnostic decision variable as a sum of three uncorrelated components that represent differences among cases, readers, and repeated readings by a given reader. This model provides a basis for quantitative predictions concerning the amount by which diagnostic accuracy, as specified by ROC analysis, can be enhanced by the replication of image readings. Although these predictions apply exactly only to the hypothetical situation in which normally distributed decision variables from equivalent readers are averaged, computer-simulation studies and an analysis of mammographic image-reading data from five radiologists show that similar gains in accuracy can be achieved by averaging discrete confidence ratings.